[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:14:15.848084 10252 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.3.62:44419
I20260709 13:14:15.849753 10252 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:14:15.850924 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:15.868458 10261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:15.868489 10259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:15.870008 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:15.869553 10258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.063607 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:17.063925 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:17.064149 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602857064124 us; error 0 us; skew 500 ppm
I20260709 13:14:17.065006 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:17.071563 10252 webserver.cc:533] Webserver started at http://127.10.3.62:46503/ using document root <none> and password file <none>
I20260709 13:14:17.072464 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:17.072712 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:17.073180 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:17.079071 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/master-0-root/instance:
uuid: "0090722fbc194c6e919a826e4d6020e6"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.086849 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:14:17.091995 10267 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.093195 10252 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:14:17.093590 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/master-0-root
uuid: "0090722fbc194c6e919a826e4d6020e6"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.093953 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:17.148003 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:17.149570 10252 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:14:17.150058 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:17.223865 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.62:44419
I20260709 13:14:17.223970 10318 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.62:44419 every 8 connection(s)
I20260709 13:14:17.232256 10319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:17.256467 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: Bootstrap starting.
I20260709 13:14:17.262898 10319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:17.265021 10319 log.cc:826] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: Log is configured to *not* fsync() on all Append() calls
I20260709 13:14:17.269963 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: No bootstrap required, opened a new log
I20260709 13:14:17.293116 10319 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER }
I20260709 13:14:17.293748 10319 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:17.293972 10319 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0090722fbc194c6e919a826e4d6020e6, State: Initialized, Role: FOLLOWER
I20260709 13:14:17.295080 10319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER }
I20260709 13:14:17.295607 10319 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:17.295885 10319 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:17.296212 10319 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:17.300693 10319 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER }
I20260709 13:14:17.301446 10319 leader_election.cc:304] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0090722fbc194c6e919a826e4d6020e6; no voters: 
I20260709 13:14:17.303309 10319 leader_election.cc:290] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:17.303654 10322 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:17.306595 10322 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 1 LEADER]: Becoming Leader. State: Replica: 0090722fbc194c6e919a826e4d6020e6, State: Running, Role: LEADER
I20260709 13:14:17.307646 10322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER }
I20260709 13:14:17.308835 10319 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:17.320132 10324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0090722fbc194c6e919a826e4d6020e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER } }
I20260709 13:14:17.321127 10324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:17.321188 10323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0090722fbc194c6e919a826e4d6020e6. Latest consensus state: current_term: 1 leader_uuid: "0090722fbc194c6e919a826e4d6020e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0090722fbc194c6e919a826e4d6020e6" member_type: VOTER } }
I20260709 13:14:17.321859 10323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:17.324661 10331 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:17.339195 10331 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:17.341638 10252 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:17.358328 10331 catalog_manager.cc:1383] Generated new cluster ID: 8bc69e2614d44c158644893bda2f3f5a
I20260709 13:14:17.358650 10331 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:17.388664 10331 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:17.390259 10331 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:17.409147 10331 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: Generated new TSK 0
I20260709 13:14:17.410215 10331 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:17.472795 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:17.481354 10341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:17.482479 10342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:17.486709 10344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.486792 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:17.487818 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:17.488018 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:17.488258 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602857488219 us; error 0 us; skew 500 ppm
I20260709 13:14:17.488897 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:17.493422 10252 webserver.cc:533] Webserver started at http://127.10.3.1:43271/ using document root <none> and password file <none>
I20260709 13:14:17.494012 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:17.494221 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:17.494519 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:17.495828 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-0-root/instance:
uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.501614 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:14:17.506214 10349 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.507125 10252 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:14:17.507419 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.507685 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:17.535280 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:17.536716 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:17.539578 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:17.544339 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:17.544577 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.544905 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:17.545121 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.652820 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.1:42701
I20260709 13:14:17.652962 10411 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.1:42701 every 8 connection(s)
I20260709 13:14:17.660539 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:17.669517 10416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:17.672648 10417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.677649 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:17.678143 10420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.679419 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:17.679709 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:17.679944 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602857679923 us; error 0 us; skew 500 ppm
I20260709 13:14:17.680578 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:17.683740 10252 webserver.cc:533] Webserver started at http://127.10.3.2:35213/ using document root <none> and password file <none>
I20260709 13:14:17.684422 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:17.684684 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:17.684999 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:17.686287 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-1-root/instance:
uuid: "3884684625ea4fd58eae56cf80373794"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.691766 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.006s
I20260709 13:14:17.697564 10424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.698542 10252 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:14:17.698881 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
uuid: "3884684625ea4fd58eae56cf80373794"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.699369 10412 heartbeater.cc:344] Connected to a master server at 127.10.3.62:44419
I20260709 13:14:17.699261 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:17.699844 10412 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:17.701187 10412 heartbeater.cc:507] Master 127.10.3.62:44419 requested a full tablet report, sending...
I20260709 13:14:17.705030 10284 ts_manager.cc:194] Registered new tserver with Master: 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701)
I20260709 13:14:17.708196 10284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55788
I20260709 13:14:17.721402 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:17.722755 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:17.724920 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:17.728042 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:17.728269 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.728543 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:17.728724 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.823938 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.2:44061
I20260709 13:14:17.824020 10486 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.2:44061 every 8 connection(s)
I20260709 13:14:17.829319 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:17.838433 10490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:17.840798 10491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.842537 10487 heartbeater.cc:344] Connected to a master server at 127.10.3.62:44419
I20260709 13:14:17.842952 10487 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:17.843870 10487 heartbeater.cc:507] Master 127.10.3.62:44419 requested a full tablet report, sending...
I20260709 13:14:17.846235 10284 ts_manager.cc:194] Registered new tserver with Master: 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:17.849349 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:17.849306 10284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55800
W20260709 13:14:17.851068 10493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:17.852183 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:17.852440 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:17.852644 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602857852626 us; error 0 us; skew 500 ppm
I20260709 13:14:17.853368 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:17.856104 10252 webserver.cc:533] Webserver started at http://127.10.3.3:33103/ using document root <none> and password file <none>
I20260709 13:14:17.856738 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:17.856968 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:17.857275 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:17.858599 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-2-root/instance:
uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.864179 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:17.868355 10498 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.869297 10252 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:14:17.869599 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
format_stamp: "Formatted at 2026-07-09 13:14:17 on dist-test-slave-gvjr"
I20260709 13:14:17.869908 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602855680335-10252-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:17.898815 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:17.900105 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:17.902397 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:17.905275 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:17.905514 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.905768 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:17.905946 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:17.997743 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.3:42953
I20260709 13:14:17.997944 10560 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.3:42953 every 8 connection(s)
I20260709 13:14:18.016644 10561 heartbeater.cc:344] Connected to a master server at 127.10.3.62:44419
I20260709 13:14:18.017086 10561 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:18.018054 10561 heartbeater.cc:507] Master 127.10.3.62:44419 requested a full tablet report, sending...
I20260709 13:14:18.020478 10284 ts_manager.cc:194] Registered new tserver with Master: 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:18.020701 10252 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015041498s
I20260709 13:14:18.022233 10284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55806
I20260709 13:14:18.071285 10284 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55822:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:18.076246 10284 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:18.190944 10526 tablet_service.cc:1511] Processing CreateTablet for tablet d7aef538eb944672a1755ead81eae573 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:18.192265 10452 tablet_service.cc:1511] Processing CreateTablet for tablet d7aef538eb944672a1755ead81eae573 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:18.192356 10377 tablet_service.cc:1511] Processing CreateTablet for tablet d7aef538eb944672a1755ead81eae573 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:14:18.193156 10526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aef538eb944672a1755ead81eae573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.193943 10377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aef538eb944672a1755ead81eae573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.194024 10452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aef538eb944672a1755ead81eae573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.196241 10451 tablet_service.cc:1511] Processing CreateTablet for tablet 61c258b6385e4d63a206c4c011086ad9 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:18.196702 10376 tablet_service.cc:1511] Processing CreateTablet for tablet 61c258b6385e4d63a206c4c011086ad9 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:18.198465 10449 tablet_service.cc:1511] Processing CreateTablet for tablet ac214c4415ba45b9a122b5b5b14cf0e2 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:18.199148 10375 tablet_service.cc:1511] Processing CreateTablet for tablet b702b1851a79469391352d39a2ea1ce8 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:18.199723 10524 tablet_service.cc:1511] Processing CreateTablet for tablet b702b1851a79469391352d39a2ea1ce8 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:18.211200 10448 tablet_service.cc:1511] Processing CreateTablet for tablet d0377ed34d4a497683fd25ab2179b8da (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:18.211660 10524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b702b1851a79469391352d39a2ea1ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.218014 10376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c258b6385e4d63a206c4c011086ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.223160 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c258b6385e4d63a206c4c011086ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.228889 10448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0377ed34d4a497683fd25ab2179b8da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.229487 10522 tablet_service.cc:1511] Processing CreateTablet for tablet d0377ed34d4a497683fd25ab2179b8da (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:18.227612 10450 tablet_service.cc:1511] Processing CreateTablet for tablet b702b1851a79469391352d39a2ea1ce8 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:14:18.229962 10446 tablet_service.cc:1511] Processing CreateTablet for tablet b764787ab3314c65a5a309233c95f29b (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:18.216760 10447 tablet_service.cc:1511] Processing CreateTablet for tablet 8a649d4e43914f32864c4d69759c75e1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:18.233448 10375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b702b1851a79469391352d39a2ea1ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.239020 10449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac214c4415ba45b9a122b5b5b14cf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.239111 10445 tablet_service.cc:1511] Processing CreateTablet for tablet b999ae81d275456e856cc7df04961eb1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:18.241904 10522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0377ed34d4a497683fd25ab2179b8da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.241807 10520 tablet_service.cc:1511] Processing CreateTablet for tablet b764787ab3314c65a5a309233c95f29b (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:18.235226 10523 tablet_service.cc:1511] Processing CreateTablet for tablet ac214c4415ba45b9a122b5b5b14cf0e2 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:18.241690 10370 tablet_service.cc:1511] Processing CreateTablet for tablet b999ae81d275456e856cc7df04961eb1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:18.237388 10519 tablet_service.cc:1511] Processing CreateTablet for tablet b999ae81d275456e856cc7df04961eb1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:14:18.242393 10369 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:18.249045 10369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.242578 10373 tablet_service.cc:1511] Processing CreateTablet for tablet d0377ed34d4a497683fd25ab2179b8da (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:14:18.249653 10521 tablet_service.cc:1511] Processing CreateTablet for tablet 8a649d4e43914f32864c4d69759c75e1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:18.251732 10444 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:18.251752 10450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b702b1851a79469391352d39a2ea1ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.235926 10374 tablet_service.cc:1511] Processing CreateTablet for tablet ac214c4415ba45b9a122b5b5b14cf0e2 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:14:18.249745 10371 tablet_service.cc:1511] Processing CreateTablet for tablet b764787ab3314c65a5a309233c95f29b (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:14:18.254242 10372 tablet_service.cc:1511] Processing CreateTablet for tablet 8a649d4e43914f32864c4d69759c75e1 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:14:18.257009 10520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b764787ab3314c65a5a309233c95f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.251782 10518 tablet_service.cc:1511] Processing CreateTablet for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:14:18.247387 10525 tablet_service.cc:1511] Processing CreateTablet for tablet 61c258b6385e4d63a206c4c011086ad9 (DEFAULT_TABLE table=test-workload [id=a7de53a96b6241cba049a0f83805af13]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:14:18.266739 10445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b999ae81d275456e856cc7df04961eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.264765 10370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b999ae81d275456e856cc7df04961eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.269495 10446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b764787ab3314c65a5a309233c95f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.273128 10523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac214c4415ba45b9a122b5b5b14cf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.273964 10444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.274050 10374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac214c4415ba45b9a122b5b5b14cf0e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.277158 10371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b764787ab3314c65a5a309233c95f29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.277500 10518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c5dbfb73bbc469a8c28e69d559c4a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.279456 10372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a649d4e43914f32864c4d69759c75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.283383 10447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a649d4e43914f32864c4d69759c75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.287082 10521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a649d4e43914f32864c4d69759c75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.291083 10519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b999ae81d275456e856cc7df04961eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.296615 10525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c258b6385e4d63a206c4c011086ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.292937 10373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0377ed34d4a497683fd25ab2179b8da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:18.352630 10332 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:18.493327 10581 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.501394 10579 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:18.528728 10580 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.548370 10581 tablet_bootstrap.cc:654] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.550328 10580 tablet_bootstrap.cc:654] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.557899 10579 tablet_bootstrap.cc:654] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.563809 10580 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.565543 10581 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.564316 10580 ts_tablet_manager.cc:1403] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.056s	user 0.004s	sys 0.008s
I20260709 13:14:18.565938 10579 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:18.566047 10581 ts_tablet_manager.cc:1403] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.016s
I20260709 13:14:18.566530 10579 ts_tablet_manager.cc:1403] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.076s	user 0.009s	sys 0.007s
I20260709 13:14:18.569160 10581 raft_consensus.cc:359] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.569470 10580 raft_consensus.cc:359] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.569953 10581 raft_consensus.cc:385] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.569646 10579 raft_consensus.cc:359] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.570233 10580 raft_consensus.cc:385] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.570392 10581 raft_consensus.cc:740] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.570458 10579 raft_consensus.cc:385] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.570598 10580 raft_consensus.cc:740] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.571082 10579 raft_consensus.cc:740] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.571339 10581 consensus_queue.cc:260] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.571525 10580 consensus_queue.cc:260] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.572525 10579 consensus_queue.cc:260] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.577075 10487 heartbeater.cc:499] Master 127.10.3.62:44419 was elected leader, sending a full tablet report...
I20260709 13:14:18.578538 10579 ts_tablet_manager.cc:1434] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
W20260709 13:14:18.579252 10488 tablet.cc:2406] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:14:18.580061 10579 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:18.588270 10412 heartbeater.cc:499] Master 127.10.3.62:44419 was elected leader, sending a full tablet report...
I20260709 13:14:18.588680 10581 ts_tablet_manager.cc:1434] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.005s
I20260709 13:14:18.589264 10561 heartbeater.cc:499] Master 127.10.3.62:44419 was elected leader, sending a full tablet report...
I20260709 13:14:18.589581 10581 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.593677 10579 tablet_bootstrap.cc:654] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.596184 10581 tablet_bootstrap.cc:654] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.604593 10580 ts_tablet_manager.cc:1434] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.038s	user 0.011s	sys 0.005s
I20260709 13:14:18.605937 10580 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.611678 10579 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:18.612218 10579 ts_tablet_manager.cc:1403] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.032s	user 0.024s	sys 0.000s
I20260709 13:14:18.614281 10580 tablet_bootstrap.cc:654] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.616770 10579 raft_consensus.cc:359] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.619206 10579 raft_consensus.cc:385] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.620560 10579 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.621347 10579 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.624111 10581 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.625326 10581 ts_tablet_manager.cc:1403] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.036s	user 0.002s	sys 0.016s
I20260709 13:14:18.628221 10581 raft_consensus.cc:359] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.629940 10581 raft_consensus.cc:385] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.630273 10581 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.631012 10581 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.631745 10579 ts_tablet_manager.cc:1434] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:14:18.633103 10579 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:18.637914 10581 ts_tablet_manager.cc:1434] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:14:18.638935 10581 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.645016 10581 tablet_bootstrap.cc:654] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.648236 10579 tablet_bootstrap.cc:654] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.687459 10581 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.689015 10580 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.689450 10580 ts_tablet_manager.cc:1403] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.084s	user 0.023s	sys 0.053s
I20260709 13:14:18.689581 10579 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:18.690012 10579 ts_tablet_manager.cc:1403] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.012s
I20260709 13:14:18.691782 10580 raft_consensus.cc:359] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.692341 10580 raft_consensus.cc:385] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.692598 10580 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.692627 10579 raft_consensus.cc:359] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.688148 10581 ts_tablet_manager.cc:1403] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.014s
I20260709 13:14:18.693362 10579 raft_consensus.cc:385] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.694006 10579 raft_consensus.cc:740] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.693354 10580 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.696210 10580 ts_tablet_manager.cc:1434] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:18.696750 10581 raft_consensus.cc:359] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.697196 10580 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.697445 10581 raft_consensus.cc:385] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.697804 10581 raft_consensus.cc:740] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.698452 10581 consensus_queue.cc:260] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.700762 10581 ts_tablet_manager.cc:1434] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:14:18.702029 10581 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.704583 10580 tablet_bootstrap.cc:654] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.707998 10581 tablet_bootstrap.cc:654] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.716187 10580 tablet_bootstrap.cc:492] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.716712 10580 ts_tablet_manager.cc:1403] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.012s
I20260709 13:14:18.719547 10580 raft_consensus.cc:359] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.720300 10580 raft_consensus.cc:385] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.720587 10580 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.721344 10580 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.723965 10580 ts_tablet_manager.cc:1434] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:14:18.724910 10580 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.727711 10581 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.728163 10581 ts_tablet_manager.cc:1403] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.015s
I20260709 13:14:18.731145 10580 tablet_bootstrap.cc:654] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.730736 10581 raft_consensus.cc:359] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.731535 10581 raft_consensus.cc:385] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.731837 10581 raft_consensus.cc:740] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.732488 10581 consensus_queue.cc:260] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.734838 10581 ts_tablet_manager.cc:1434] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:18.735862 10581 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.742398 10581 tablet_bootstrap.cc:654] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.746481 10579 consensus_queue.cc:260] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.749891 10581 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.750403 10581 ts_tablet_manager.cc:1403] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:14:18.751120 10580 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.751587 10580 ts_tablet_manager.cc:1403] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.015s
I20260709 13:14:18.753173 10581 raft_consensus.cc:359] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.753893 10581 raft_consensus.cc:385] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.754185 10581 raft_consensus.cc:740] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.754413 10580 raft_consensus.cc:359] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.755311 10580 raft_consensus.cc:385] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.754871 10581 consensus_queue.cc:260] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.755625 10580 raft_consensus.cc:740] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.756299 10579 ts_tablet_manager.cc:1434] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:14:18.756402 10580 consensus_queue.cc:260] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.759418 10580 ts_tablet_manager.cc:1434] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:18.760306 10580 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.757477 10581 ts_tablet_manager.cc:1434] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:18.763805 10581 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.764896 10579 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:18.769851 10580 tablet_bootstrap.cc:654] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.770750 10586 raft_consensus.cc:493] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.771272 10586 raft_consensus.cc:515] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.772511 10579 tablet_bootstrap.cc:654] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.774245 10581 tablet_bootstrap.cc:654] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.775640 10586 leader_election.cc:290] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:18.776180 10586 raft_consensus.cc:493] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.776651 10586 raft_consensus.cc:515] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.779116 10586 leader_election.cc:290] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:18.794549 10580 tablet_bootstrap.cc:492] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.795289 10580 ts_tablet_manager.cc:1403] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.002s
I20260709 13:14:18.798935 10580 raft_consensus.cc:359] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.800182 10580 raft_consensus.cc:385] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.800704 10580 raft_consensus.cc:740] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.801720 10580 consensus_queue.cc:260] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.804814 10580 ts_tablet_manager.cc:1434] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:14:18.806046 10580 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.814436 10580 tablet_bootstrap.cc:654] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.832975 10587 raft_consensus.cc:493] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.833592 10587 raft_consensus.cc:515] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.838088 10587 leader_election.cc:290] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:18.882828 10581 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:18.883396 10581 ts_tablet_manager.cc:1403] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.120s	user 0.027s	sys 0.030s
I20260709 13:14:18.886495 10581 raft_consensus.cc:359] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.887287 10581 raft_consensus.cc:385] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.887600 10581 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.889654 10585 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.890134 10585 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:18.893013 10585 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:18.892244 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:18.890337 10581 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.896278 10581 ts_tablet_manager.cc:1434] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:14:18.897217 10581 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.903409 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
W20260709 13:14:18.906002 10501 leader_election.cc:343] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:18.906260 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:18.911339 10585 raft_consensus.cc:493] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.911840 10585 raft_consensus.cc:515] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.913844 10585 leader_election.cc:290] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
W20260709 13:14:18.915616 10501 leader_election.cc:343] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:18.918256 10581 tablet_bootstrap.cc:654] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.919799 10579 tablet_bootstrap.cc:492] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:18.920470 10579 ts_tablet_manager.cc:1403] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.156s	user 0.016s	sys 0.002s
I20260709 13:14:18.924257 10579 raft_consensus.cc:359] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.931262 10581 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
W20260709 13:14:18.931145 10426 leader_election.cc:343] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:18.934886 10581 ts_tablet_manager.cc:1403] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.012s
I20260709 13:14:18.935883 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:18.936592 10387 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
I20260709 13:14:18.938234 10580 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:18.938699 10580 ts_tablet_manager.cc:1403] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.133s	user 0.028s	sys 0.024s
I20260709 13:14:18.938808 10579 raft_consensus.cc:385] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.939254 10579 raft_consensus.cc:740] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.939437 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:18.940099 10579 consensus_queue.cc:260] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.941753 10580 raft_consensus.cc:359] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.942454 10580 raft_consensus.cc:385] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.942628 10579 ts_tablet_manager.cc:1434] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 13:14:18.940771 10500 leader_election.cc:304] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 3884684625ea4fd58eae56cf80373794
I20260709 13:14:18.943701 10579 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:18.944137 10598 raft_consensus.cc:2804] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:18.944538 10598 raft_consensus.cc:493] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.944868 10598 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:18.938722 10581 raft_consensus.cc:359] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.948026 10581 raft_consensus.cc:385] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.948343 10581 raft_consensus.cc:740] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.949101 10581 consensus_queue.cc:260] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.950129 10387 raft_consensus.cc:2468] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
I20260709 13:14:18.951036 10536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:18.942771 10580 raft_consensus.cc:740] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
W20260709 13:14:18.956110 10425 leader_election.cc:343] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:18.956718 10425 leader_election.cc:304] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3884684625ea4fd58eae56cf80373794; no voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:18.957751 10587 raft_consensus.cc:2749] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:18.957125 10581 ts_tablet_manager.cc:1434] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:14:18.959262 10580 consensus_queue.cc:260] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:18.963651 10579 tablet_bootstrap.cc:654] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.964419 10500 leader_election.cc:304] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 3884684625ea4fd58eae56cf80373794
I20260709 13:14:18.969126 10581 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:18.970137 10580 ts_tablet_manager.cc:1434] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:14:18.971531 10586 raft_consensus.cc:2804] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:18.971879 10586 raft_consensus.cc:493] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.972211 10586 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:18.989482 10580 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:18.997654 10585 raft_consensus.cc:493] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.998976 10585 raft_consensus.cc:515] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.001955 10585 leader_election.cc:290] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.002132 10586 raft_consensus.cc:515] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.003791 10579 tablet_bootstrap.cc:492] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:19.004279 10579 ts_tablet_manager.cc:1403] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.005s
I20260709 13:14:19.008051 10585 raft_consensus.cc:493] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.008283 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.008589 10585 raft_consensus.cc:515] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.008859 10387 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:18.994006 10598 raft_consensus.cc:515] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.009009 10586 leader_election.cc:290] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.010669 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.011605 10581 tablet_bootstrap.cc:654] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:19.015265 10501 leader_election.cc:343] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.016796 10585 leader_election.cc:290] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.012511 10586 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.017865 10586 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.017868 10579 raft_consensus.cc:359] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.018754 10579 raft_consensus.cc:385] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.020097 10536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.019093 10579 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.020715 10536 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.021538 10586 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.022771 10386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:19.023470 10587 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.024150 10386 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
I20260709 13:14:19.024420 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.025281 10500 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.025552 10536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.025269 10587 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.023599 10579 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.026275 10586 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.027264 10461 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
I20260709 13:14:19.029400 10587 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953), 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701)
I20260709 13:14:19.026903 10586 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.027511 10536 raft_consensus.cc:2393] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8eb3bf571c5a43e686ae4dc869c789f9 in current term 1: Already voted for candidate 9d248dbd1e694e3fb2ada1133206ab5a in this term.
I20260709 13:14:19.030543 10586 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.032801 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.031427 10580 tablet_bootstrap.cc:654] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.039192 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.041496 10350 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.041720 10534 raft_consensus.cc:2468] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.042928 10585 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.039942 10533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.045903 10585 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.046603 10585 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.050383 10350 leader_election.cc:304] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.043309 10386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:19.050549 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.054068 10616 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.055560 10616 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.057749 10579 ts_tablet_manager.cc:1434] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.052s	user 0.013s	sys 0.008s
I20260709 13:14:19.058065 10587 raft_consensus.cc:493] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.058486 10587 raft_consensus.cc:515] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.058919 10579 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:19.060516 10587 leader_election.cc:290] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.053597 10385 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.053187 10387 raft_consensus.cc:2468] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.077184 10384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:19.078573 10384 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
I20260709 13:14:19.078799 10500 leader_election.cc:304] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.080772 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.069976 10533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.067922 10598 leader_election.cc:290] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.134361 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.081462 10461 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.088843 10598 raft_consensus.cc:2804] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.130249 10616 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.174732 10457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.129155 10585 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.174630 10598 raft_consensus.cc:697] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.180006 10585 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Requested vote from peers 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.180594 10585 raft_consensus.cc:2804] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.180951 10585 raft_consensus.cc:493] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.181296 10585 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.181948 10586 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.115217 10533 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 0.
I20260709 13:14:19.183240 10386 raft_consensus.cc:2393] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3884684625ea4fd58eae56cf80373794 in current term 1: Already voted for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in this term.
I20260709 13:14:19.175729 10462 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 0.
W20260709 13:14:19.175877 10500 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:42701, user_credentials={real_user=slave}} blocked reactor thread for 97169us
I20260709 13:14:19.186554 10535 raft_consensus.cc:2393] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3884684625ea4fd58eae56cf80373794 in current term 1: Already voted for candidate 9d248dbd1e694e3fb2ada1133206ab5a in this term.
I20260709 13:14:19.138334 10458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.182482 10598 consensus_queue.cc:237] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.191898 10581 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:19.192623 10581 ts_tablet_manager.cc:1403] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.227s	user 0.062s	sys 0.078s
I20260709 13:14:19.138085 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.136092 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
W20260709 13:14:19.196197 10352 leader_election.cc:343] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.199179 10586 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
W20260709 13:14:19.198625 10350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:42953, user_credentials={real_user=slave}} blocked reactor thread for 150728us
W20260709 13:14:19.202383 10352 leader_election.cc:343] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.203783 10386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:19.207640 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.208209 10386 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 0.
I20260709 13:14:19.204003 10460 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.209379 10385 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.212327 10580 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:19.214143 10579 tablet_bootstrap.cc:654] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:19.203678 10350 leader_election.cc:343] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.195504 10500 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.216588 10580 ts_tablet_manager.cc:1403] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.236s	user 0.018s	sys 0.004s
I20260709 13:14:19.209059 10533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.218358 10350 leader_election.cc:304] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.219870 10425 leader_election.cc:304] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.220278 10352 leader_election.cc:304] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.221587 10617 raft_consensus.cc:2804] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.222381 10617 raft_consensus.cc:493] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.222718 10617 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.224217 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.227556 10459 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.219269 10586 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.228643 10586 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.229056 10586 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.228766 10425 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3884684625ea4fd58eae56cf80373794; no voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.231520 10533 raft_consensus.cc:2393] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8eb3bf571c5a43e686ae4dc869c789f9 in current term 1: Already voted for candidate 9d248dbd1e694e3fb2ada1133206ab5a in this term.
I20260709 13:14:19.238837 10350 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.240144 10580 raft_consensus.cc:359] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.240856 10580 raft_consensus.cc:385] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.241178 10580 raft_consensus.cc:740] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.241604 10585 raft_consensus.cc:515] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.242056 10580 consensus_queue.cc:260] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.241159 10461 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.245050 10585 leader_election.cc:290] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.245550 10585 raft_consensus.cc:2749] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.246002 10581 raft_consensus.cc:359] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.246627 10581 raft_consensus.cc:385] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.246927 10581 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.246075 10580 ts_tablet_manager.cc:1434] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.004s
I20260709 13:14:19.247605 10581 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.233912 10500 leader_election.cc:304] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.248013 10585 raft_consensus.cc:2749] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.250790 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.252218 10385 raft_consensus.cc:2393] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9d248dbd1e694e3fb2ada1133206ab5a in current term 1: Already voted for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in this term.
I20260709 13:14:19.254292 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.256057 10598 raft_consensus.cc:2804] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.248304 10619 raft_consensus.cc:493] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.263176 10598 raft_consensus.cc:697] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.265259 10581 ts_tablet_manager.cc:1434] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.001s
I20260709 13:14:19.265067 10598 consensus_queue.cc:237] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.266180 10581 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: Bootstrap starting.
I20260709 13:14:19.269580 10623 raft_consensus.cc:2749] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:14:19.263175 10352 leader_election.cc:343] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.267603 10617 raft_consensus.cc:515] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.279446 10533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.280027 10533 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.280619 10459 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.277676 10580 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:19.282835 10587 raft_consensus.cc:2749] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.282176 10501 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:19.283946 10598 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.284392 10598 raft_consensus.cc:697] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.285476 10598 consensus_queue.cc:237] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.287284 10586 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.257439 10619 raft_consensus.cc:515] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.281417 10282 catalog_manager.cc:5697] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.286729 10617 leader_election.cc:290] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.290899 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.291515 10535 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.292912 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.293849 10385 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.307003 10385 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 1.
I20260709 13:14:19.308200 10535 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 1.
I20260709 13:14:19.308617 10426 leader_election.cc:304] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.309744 10617 raft_consensus.cc:2804] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.310387 10617 raft_consensus.cc:697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:19.313369 10617 consensus_queue.cc:237] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.313866 10533 raft_consensus.cc:2468] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 1.
I20260709 13:14:19.315706 10619 leader_election.cc:290] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.315187 10350 leader_election.cc:304] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.317348 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.319114 10619 raft_consensus.cc:2804] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.320907 10619 raft_consensus.cc:697] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.327056 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.327891 10534 raft_consensus.cc:2393] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8eb3bf571c5a43e686ae4dc869c789f9 in current term 1: Already voted for candidate 3884684625ea4fd58eae56cf80373794 in this term.
I20260709 13:14:19.329411 10579 tablet_bootstrap.cc:492] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:19.329789 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.329739 10350 leader_election.cc:304] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.329914 10579 ts_tablet_manager.cc:1403] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.271s	user 0.024s	sys 0.022s
I20260709 13:14:19.323029 10619 consensus_queue.cc:237] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.334707 10581 tablet_bootstrap.cc:654] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.336014 10625 raft_consensus.cc:2749] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.338634 10586 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.333328 10385 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.339823 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.340413 10459 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.340384 10580 tablet_bootstrap.cc:654] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.349525 10579 raft_consensus.cc:359] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.350274 10579 raft_consensus.cc:385] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.350567 10579 raft_consensus.cc:740] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.351326 10385 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.352450 10500 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.353408 10586 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.352980 10579 consensus_queue.cc:260] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.357074 10586 raft_consensus.cc:697] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.358124 10586 consensus_queue.cc:237] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.360119 10459 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 1.
I20260709 13:14:19.363232 10579 ts_tablet_manager.cc:1434] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.005s
I20260709 13:14:19.370131 10579 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
W20260709 13:14:19.372081 10332 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:19.371677 10281 catalog_manager.cc:5697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.372617 10283 catalog_manager.cc:5697] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.382835 10579 tablet_bootstrap.cc:654] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.383972 10581 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: No bootstrap required, opened a new log
I20260709 13:14:19.384466 10581 ts_tablet_manager.cc:1403] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.013s
I20260709 13:14:19.387166 10580 tablet_bootstrap.cc:492] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:19.387610 10580 ts_tablet_manager.cc:1403] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.113s	user 0.011s	sys 0.005s
I20260709 13:14:19.390218 10580 raft_consensus.cc:359] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.390857 10580 raft_consensus.cc:385] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.391170 10580 raft_consensus.cc:740] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.391849 10580 consensus_queue.cc:260] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.394220 10579 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:19.394728 10579 ts_tablet_manager.cc:1403] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.006s
I20260709 13:14:19.396826 10581 raft_consensus.cc:359] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.397534 10581 raft_consensus.cc:385] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.397518 10579 raft_consensus.cc:359] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.398152 10581 raft_consensus.cc:740] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.398298 10579 raft_consensus.cc:385] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.398765 10579 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.398977 10581 consensus_queue.cc:260] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.399693 10579 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.401257 10581 ts_tablet_manager.cc:1434] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:14:19.402115 10579 ts_tablet_manager.cc:1434] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:19.402945 10580 ts_tablet_manager.cc:1434] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:14:19.403126 10579 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:19.403842 10580 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: Bootstrap starting.
I20260709 13:14:19.398383 10282 catalog_manager.cc:5697] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.409945 10282 catalog_manager.cc:5697] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.411146 10580 tablet_bootstrap.cc:654] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.412081 10579 tablet_bootstrap.cc:654] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.415199 10282 catalog_manager.cc:5697] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.461440 10580 tablet_bootstrap.cc:492] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: No bootstrap required, opened a new log
I20260709 13:14:19.461879 10580 ts_tablet_manager.cc:1403] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.033s
I20260709 13:14:19.462373 10579 tablet_bootstrap.cc:492] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:19.462802 10579 ts_tablet_manager.cc:1403] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.021s
I20260709 13:14:19.463872 10580 raft_consensus.cc:359] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.464375 10580 raft_consensus.cc:385] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.464592 10580 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.465204 10580 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.465176 10579 raft_consensus.cc:359] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.465965 10579 raft_consensus.cc:385] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.466251 10579 raft_consensus.cc:740] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.467173 10580 ts_tablet_manager.cc:1434] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:14:19.466882 10579 consensus_queue.cc:260] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.469072 10579 ts_tablet_manager.cc:1434] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:19.469938 10579 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: Bootstrap starting.
I20260709 13:14:19.475441 10579 tablet_bootstrap.cc:654] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.480154 10579 tablet_bootstrap.cc:492] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: No bootstrap required, opened a new log
I20260709 13:14:19.480590 10579 ts_tablet_manager.cc:1403] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:14:19.483090 10579 raft_consensus.cc:359] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.483855 10579 raft_consensus.cc:385] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.484227 10579 raft_consensus.cc:740] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.485056 10579 consensus_queue.cc:260] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.487080 10579 ts_tablet_manager.cc:1434] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:14:19.514081 10617 raft_consensus.cc:493] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.514613 10617 raft_consensus.cc:515] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.516726 10617 leader_election.cc:290] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.517434 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" is_pre_election: true
I20260709 13:14:19.517784 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.517990 10385 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 0.
I20260709 13:14:19.518410 10534 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 0.
I20260709 13:14:19.519018 10426 leader_election.cc:304] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.519796 10617 raft_consensus.cc:2804] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.520146 10617 raft_consensus.cc:493] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.520422 10617 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.525938 10617 raft_consensus.cc:515] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.527956 10617 leader_election.cc:290] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.528410 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:19.528954 10385 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.528980 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.529656 10534 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.535765 10385 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 1.
I20260709 13:14:19.536096 10534 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 1.
I20260709 13:14:19.536547 10619 raft_consensus.cc:493] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.537001 10619 raft_consensus.cc:515] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.537393 10425 leader_election.cc:304] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.538163 10617 raft_consensus.cc:2804] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.538817 10619 leader_election.cc:290] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.538875 10617 raft_consensus.cc:697] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:19.540115 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.539927 10617 consensus_queue.cc:237] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.540421 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.540721 10459 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.541039 10534 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.541956 10352 leader_election.cc:304] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.543056 10619 raft_consensus.cc:2804] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.543465 10619 raft_consensus.cc:493] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.543877 10619 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.548696 10283 catalog_manager.cc:5697] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.549836 10619 raft_consensus.cc:515] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.553179 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.553404 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.553965 10459 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.554108 10534 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.560693 10459 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 1.
I20260709 13:14:19.560680 10534 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 1.
I20260709 13:14:19.560801 10619 leader_election.cc:290] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.561883 10352 leader_election.cc:304] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.562690 10619 raft_consensus.cc:2804] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.563143 10619 raft_consensus.cc:697] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.563930 10619 consensus_queue.cc:237] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.571374 10283 catalog_manager.cc:5697] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.582593 10586 consensus_queue.cc:1048] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:19.584939 10619 raft_consensus.cc:493] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.585426 10619 raft_consensus.cc:515] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.588202 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794" is_pre_election: true
I20260709 13:14:19.588817 10459 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.588963 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" is_pre_election: true
I20260709 13:14:19.590273 10534 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 0.
I20260709 13:14:19.591389 10350 leader_election.cc:304] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.592206 10625 raft_consensus.cc:2804] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.589417 10619 leader_election.cc:290] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.592559 10625 raft_consensus.cc:493] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.592927 10625 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.597594 10459 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.601483 10625 raft_consensus.cc:515] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.605120 10461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.605882 10461 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.610589 10598 consensus_queue.cc:1048] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.613284 10625 leader_election.cc:290] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.614357 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.614591 10461 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 1.
I20260709 13:14:19.614946 10534 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.617588 10352 leader_election.cc:304] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.618506 10625 raft_consensus.cc:2804] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.619832 10625 raft_consensus.cc:697] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.620865 10625 consensus_queue.cc:237] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.621491 10534 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 1.
I20260709 13:14:19.631915 10283 catalog_manager.cc:5697] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.719122 10534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:19.719739 10534 raft_consensus.cc:606] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:19.725230 10598 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.727228 10534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:19.727833 10534 raft_consensus.cc:606] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:19.732072 10598 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.740588 10620 raft_consensus.cc:993] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.741067 10620 raft_consensus.cc:1081] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.750495 10385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46530
I20260709 13:14:19.751215 10385 raft_consensus.cc:493] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.751605 10385 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.758296 10385 raft_consensus.cc:515] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.763985 10620 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.770296 10386 raft_consensus.cc:1240] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Rejecting Update request from peer 9d248dbd1e694e3fb2ada1133206ab5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:19.776932 10385 leader_election.cc:290] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.777693 10459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.778290 10459 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.778497 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.779093 10534 raft_consensus.cc:3055] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.779421 10534 raft_consensus.cc:740] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.780218 10534 consensus_queue.cc:260] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.781505 10534 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.781977 10461 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:47606
I20260709 13:14:19.782680 10461 raft_consensus.cc:606] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:19.781982 10655 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:14:19.793745 10459 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:19.794977 10352 leader_election.cc:304] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:19.796078 10625 raft_consensus.cc:2804] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.796538 10625 raft_consensus.cc:697] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.797343 10625 consensus_queue.cc:237] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.790832 10534 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:19.802858 10617 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.811714 10620 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.827796 10587 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.836081 10461 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.835851 10625 consensus_queue.cc:1048] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.846254 10282 catalog_manager.cc:5697] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 1 to 2, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.857028 10625 consensus_queue.cc:1048] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.859526 10385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:46544
I20260709 13:14:19.860152 10385 raft_consensus.cc:606] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.868984 10586 raft_consensus.cc:993] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.869524 10586 raft_consensus.cc:1081] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.877393 10587 raft_consensus.cc:993] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.877844 10587 raft_consensus.cc:1081] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:19.878620 10586 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.879499 10386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46542
I20260709 13:14:19.887912 10386 raft_consensus.cc:493] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.890978 10387 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46530
I20260709 13:14:19.892416 10534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:19.893532 10534 raft_consensus.cc:606] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.894941 10387 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.895412 10387 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.900626 10386 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.901566 10535 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:19.903489 10535 raft_consensus.cc:606] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.903422 10387 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.907436 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.907436 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.908269 10534 raft_consensus.cc:3055] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.908589 10534 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:19.909595 10387 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Requested vote from peers 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:19.911464 10534 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.912436 10534 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.916287 10386 raft_consensus.cc:515] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.919329 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:19.920044 10460 raft_consensus.cc:3055] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:19.920423 10460 raft_consensus.cc:740] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:19.921396 10460 consensus_queue.cc:260] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.922600 10460 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.924517 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:19.925474 10535 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:19.927670 10617 consensus_queue.cc:1048] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.924959 10386 leader_election.cc:290] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:19.932201 10385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:46544
I20260709 13:14:19.932840 10385 raft_consensus.cc:606] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Received request to transfer leadership to TS 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:19.940441 10535 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:19.941756 10350 leader_election.cc:304] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:19.942572 10619 raft_consensus.cc:2804] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.943048 10619 raft_consensus.cc:697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.943233 10534 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:19.943898 10619 consensus_queue.cc:237] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:19.946787 10350 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 3884684625ea4fd58eae56cf80373794
I20260709 13:14:19.948213 10460 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:19.948274 10625 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:19.960145 10625 raft_consensus.cc:697] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:19.961041 10625 consensus_queue.cc:237] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:19.973067 10617 consensus_queue.cc:1048] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:19.974262 10283 catalog_manager.cc:5697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 1 to 2, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.980362 10283 catalog_manager.cc:5697] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 1 to 2, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.998049 10667 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.015838 10625 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.053508 10667 consensus_queue.cc:1048] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:20.057663 10655 raft_consensus.cc:993] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.058111 10655 raft_consensus.cc:1081] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.057977 10667 consensus_queue.cc:1048] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.060103 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47596
I20260709 13:14:20.060703 10460 raft_consensus.cc:493] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.061077 10460 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.069159 10667 raft_consensus.cc:993] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.069656 10667 raft_consensus.cc:1081] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.073263 10459 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47604
I20260709 13:14:20.077610 10459 raft_consensus.cc:493] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.078162 10459 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.090821 10459 raft_consensus.cc:515] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.095109 10459 leader_election.cc:290] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:20.095774 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:20.096403 10387 raft_consensus.cc:3055] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.096741 10387 raft_consensus.cc:740] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:20.097534 10387 consensus_queue.cc:260] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.098729 10387 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.100860 10460 raft_consensus.cc:515] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.103502 10460 leader_election.cc:290] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:20.104776 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:20.105386 10534 raft_consensus.cc:3055] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.105157 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b764787ab3314c65a5a309233c95f29b" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:20.105746 10534 raft_consensus.cc:740] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:20.106026 10385 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.106135 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:20.106861 10535 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.106594 10534 consensus_queue.cc:260] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.107863 10534 raft_consensus.cc:3060] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.113180 10385 raft_consensus.cc:2468] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.113690 10535 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.115163 10425 leader_election.cc:304] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:20.115738 10426 leader_election.cc:304] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:20.115952 10617 raft_consensus.cc:2804] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.116535 10534 raft_consensus.cc:2468] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.116588 10587 raft_consensus.cc:2804] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.125823 10387 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.128799 10587 raft_consensus.cc:697] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:20.129503 10617 raft_consensus.cc:697] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:20.129803 10587 consensus_queue.cc:237] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.130398 10617 consensus_queue.cc:237] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.137892 10283 catalog_manager.cc:5697] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 1 to 2, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.139096 10283 catalog_manager.cc:5697] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 1 to 2, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.201005 10534 raft_consensus.cc:1275] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.202317 10619 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.210775 10460 raft_consensus.cc:1275] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.213436 10667 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.273011 10586 raft_consensus.cc:993] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.273504 10620 raft_consensus.cc:1081] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:20.275183 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47596
I20260709 13:14:20.275759 10460 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.276139 10460 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.282882 10460 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.285101 10460 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:20.286139 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:20.286139 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:20.286881 10387 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.286916 10534 raft_consensus.cc:3055] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.287513 10534 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:20.288335 10534 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.289494 10534 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.293802 10387 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.295053 10426 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:20.295862 10587 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.296039 10534 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 2.
I20260709 13:14:20.296360 10587 raft_consensus.cc:697] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:20.297210 10587 consensus_queue.cc:237] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.304816 10280 catalog_manager.cc:5697] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 1 to 2, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.396054 10460 raft_consensus.cc:1275] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.397511 10667 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.408488 10534 raft_consensus.cc:1275] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.409927 10675 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.481066 10460 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.485836 10460 raft_consensus.cc:1275] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.487195 10675 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.495034 10534 raft_consensus.cc:1275] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.496616 10625 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.501109 10667 raft_consensus.cc:993] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:20.501549 10667 raft_consensus.cc:1081] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Signalling peer 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:20.503479 10535 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:33890
I20260709 13:14:20.504109 10535 raft_consensus.cc:493] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.504498 10535 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.511272 10535 raft_consensus.cc:515] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.513492 10535 leader_election.cc:290] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 2 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:20.513831 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:20.514053 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c5dbfb73bbc469a8c28e69d559c4a3a" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:20.514518 10387 raft_consensus.cc:3055] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:20.514632 10460 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.514921 10387 raft_consensus.cc:740] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:20.515709 10387 consensus_queue.cc:260] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.516743 10387 raft_consensus.cc:3060] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:20.518376 10535 raft_consensus.cc:1275] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.519763 10587 consensus_queue.cc:1048] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.520720 10460 raft_consensus.cc:2468] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 2.
I20260709 13:14:20.521867 10501 leader_election.cc:304] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:20.522742 10586 raft_consensus.cc:2804] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:20.522828 10387 raft_consensus.cc:2468] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 2.
I20260709 13:14:20.523274 10586 raft_consensus.cc:697] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:20.523980 10586 consensus_queue.cc:237] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:20.528388 10387 raft_consensus.cc:1275] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.529719 10587 consensus_queue.cc:1048] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.539736 10279 catalog_manager.cc:5697] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 1 to 2, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:20.573374 10535 raft_consensus.cc:1275] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.574610 10617 consensus_queue.cc:1048] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.582520 10387 raft_consensus.cc:1275] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.584071 10617 consensus_queue.cc:1048] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.702129 10387 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.703390 10617 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.712212 10535 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.713708 10673 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.915112 10460 raft_consensus.cc:1275] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.916450 10586 consensus_queue.cc:1048] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:20.925925 10387 raft_consensus.cc:1275] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:20.927723 10677 consensus_queue.cc:1048] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:22.962929 10252 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:14:22.963792 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:22.984498 10387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:46546
I20260709 13:14:22.985070 10387 raft_consensus.cc:606] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Received request to transfer leadership to TS 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:22.995740 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:22.996306 10460 raft_consensus.cc:606] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Received request to transfer leadership to TS 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:22.997498 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:22.997944 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:23.317107 10617 raft_consensus.cc:993] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:23.317570 10617 raft_consensus.cc:1081] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Signalling peer 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:23.319130 10534 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:33884
I20260709 13:14:23.319633 10534 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:23.319995 10534 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:23.324622 10534 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.326241 10534 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 3 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:23.326962 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:23.327256 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:23.327616 10387 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:23.327777 10460 raft_consensus.cc:3055] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:23.328145 10460 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:23.328797 10460 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.329665 10460 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:23.332554 10387 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 3.
I20260709 13:14:23.333798 10500 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:23.334542 10680 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:23.334556 10460 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 3.
I20260709 13:14:23.335752 10680 raft_consensus.cc:697] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:23.336541 10680 consensus_queue.cc:237] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.345079 10280 catalog_manager.cc:5697] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 2 to 3, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.353698 10667 raft_consensus.cc:993] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:23.354212 10690 raft_consensus.cc:1081] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Signalling peer 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:23.355878 10534 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:33890
I20260709 13:14:23.356439 10534 raft_consensus.cc:493] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:23.356817 10534 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.361428 10534 raft_consensus.cc:515] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.363174 10534 leader_election.cc:290] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 2 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:23.363917 10387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:23.364216 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:23.364449 10387 raft_consensus.cc:3055] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:23.364773 10460 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.364802 10387 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:23.365697 10387 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.366585 10387 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:23.369454 10460 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 2.
I20260709 13:14:23.370450 10501 leader_election.cc:304] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:23.371039 10387 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 2.
I20260709 13:14:23.371170 10680 raft_consensus.cc:2804] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:23.371619 10680 raft_consensus.cc:697] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:23.372445 10680 consensus_queue.cc:237] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:23.379230 10280 catalog_manager.cc:5697] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 1 to 2, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.848227 10460 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:23.849481 10680 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:23.858191 10387 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:23.859930 10696 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:23.903600 10460 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:23.904819 10696 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:23.913228 10387 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:23.915185 10686 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:23.998708 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:24.009384 10387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:46544
I20260709 13:14:24.009928 10387 raft_consensus.cc:606] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:24.014953 10387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:46544
I20260709 13:14:24.015507 10387 raft_consensus.cc:606] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:24.020270 10534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:24.020720 10534 raft_consensus.cc:606] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:24.025403 10387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:46544
I20260709 13:14:24.026003 10387 raft_consensus.cc:606] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:24.037917 10534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
mode: GRACEFUL
new_leader_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:33902
I20260709 13:14:24.038365 10534 raft_consensus.cc:606] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Received request to transfer leadership to TS 3884684625ea4fd58eae56cf80373794
I20260709 13:14:24.165944 10702 raft_consensus.cc:993] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.166378 10690 raft_consensus.cc:1081] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.167780 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47604
I20260709 13:14:24.168206 10460 raft_consensus.cc:493] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.168443 10460 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.173097 10460 raft_consensus.cc:515] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.175202 10460 leader_election.cc:290] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:24.175578 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:24.175788 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac214c4415ba45b9a122b5b5b14cf0e2" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:24.176141 10385 raft_consensus.cc:3055] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:24.176319 10534 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.176558 10385 raft_consensus.cc:740] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:24.177301 10385 consensus_queue.cc:260] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.178179 10385 raft_consensus.cc:3060] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.180575 10534 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.181593 10425 leader_election.cc:304] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:24.182260 10617 raft_consensus.cc:2804] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:24.182825 10385 raft_consensus.cc:2468] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.183530 10617 raft_consensus.cc:697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:24.184391 10617 consensus_queue.cc:237] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.190968 10279 catalog_manager.cc:5697] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 2 to 3, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.317368 10702 raft_consensus.cc:993] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.317821 10690 raft_consensus.cc:1081] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.319425 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47604
I20260709 13:14:24.319953 10460 raft_consensus.cc:493] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.320286 10460 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.324084 10460 raft_consensus.cc:515] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.325549 10460 leader_election.cc:290] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:24.326435 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:24.326725 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c258b6385e4d63a206c4c011086ad9" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:24.327030 10385 raft_consensus.cc:3055] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:24.327327 10534 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.327435 10385 raft_consensus.cc:740] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:24.328225 10385 consensus_queue.cc:260] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.329110 10385 raft_consensus.cc:3060] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.331770 10534 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.332732 10425 leader_election.cc:304] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:24.333411 10617 raft_consensus.cc:2804] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:24.333612 10385 raft_consensus.cc:2468] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.333847 10617 raft_consensus.cc:697] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:24.335043 10617 consensus_queue.cc:237] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.341420 10279 catalog_manager.cc:5697] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 2 to 3, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.431097 10680 raft_consensus.cc:993] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.431550 10686 raft_consensus.cc:1081] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.433043 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47596
I20260709 13:14:24.433575 10460 raft_consensus.cc:493] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.434016 10460 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.440230 10460 raft_consensus.cc:515] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.441884 10460 leader_election.cc:290] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:24.442876 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:24.443152 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:24.443436 10385 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.443687 10534 raft_consensus.cc:3055] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:24.444005 10534 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:24.444658 10534 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.445454 10534 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.448319 10385 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.449546 10426 leader_election.cc:304] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:24.450399 10617 raft_consensus.cc:2804] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:24.450897 10617 raft_consensus.cc:697] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:24.450115 10534 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.451671 10617 consensus_queue.cc:237] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.454819 10680 raft_consensus.cc:993] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.455370 10680 raft_consensus.cc:1081] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.456949 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47596
I20260709 13:14:24.457445 10460 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.457746 10460 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:24.458564 10279 catalog_manager.cc:5697] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 2 to 3, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.462381 10460 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.464143 10460 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 4 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:24.465126 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:24.465302 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:24.465665 10534 raft_consensus.cc:3055] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:24.465870 10385 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:24.465991 10534 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:24.466822 10534 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.467881 10534 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:24.471058 10385 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 4.
I20260709 13:14:24.471786 10534 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 4.
I20260709 13:14:24.472225 10426 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 8eb3bf571c5a43e686ae4dc869c789f9; no voters: 
I20260709 13:14:24.472982 10617 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:24.473388 10617 raft_consensus.cc:697] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:24.474162 10617 consensus_queue.cc:237] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:24.479872 10279 catalog_manager.cc:5697] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 3 to 4, leader changed from 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 4 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.555279 10702 raft_consensus.cc:993] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: : Instructing follower 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.555748 10690 raft_consensus.cc:1081] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Signalling peer 3884684625ea4fd58eae56cf80373794 to start an election
I20260709 13:14:24.557185 10460 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
 from {username='slave'} at 127.0.0.1:47604
I20260709 13:14:24.557616 10460 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:24.557874 10460 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.561760 10460 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:24.563117 10460 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Requested vote from peers 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953), 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701)
I20260709 13:14:24.564025 10534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:24.564052 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "3884684625ea4fd58eae56cf80373794" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:24.564684 10534 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.564754 10385 raft_consensus.cc:3055] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:24.565115 10385 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:24.565882 10385 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:24.567052 10385 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:24.570868 10534 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.572149 10425 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3884684625ea4fd58eae56cf80373794, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:24.572966 10617 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:24.573423 10617 raft_consensus.cc:697] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Leader. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:24.573596 10385 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3884684625ea4fd58eae56cf80373794 in term 3.
I20260709 13:14:24.574225 10617 consensus_queue.cc:237] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:24.581620 10279 catalog_manager.cc:5697] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 reported cstate change: term changed from 2 to 3, leader changed from 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1) to 3884684625ea4fd58eae56cf80373794 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "3884684625ea4fd58eae56cf80373794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.732249 10534 raft_consensus.cc:1275] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.733531 10617 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.744289 10385 raft_consensus.cc:1275] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.745880 10703 consensus_queue.cc:1048] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.809159 10385 raft_consensus.cc:1275] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.810524 10703 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.819185 10534 raft_consensus.cc:1275] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.820937 10703 consensus_queue.cc:1048] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.857791 10534 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.859023 10703 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.868564 10385 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.869755 10711 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.983173 10385 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:24.984009 10387 raft_consensus.cc:1275] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.984536 10711 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:24.985258 10719 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:24.999238 10535 raft_consensus.cc:1275] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:24.999737 10534 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Refusing update from remote peer 3884684625ea4fd58eae56cf80373794: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:25.002699 10703 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:25.004267 10719 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:27.043005 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:27.050045 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:27.050701 10460 raft_consensus.cc:606] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:27.053258 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:27.053686 10460 raft_consensus.cc:606] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:27.055877 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:27.056269 10460 raft_consensus.cc:606] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 LEADER]: Received request to transfer leadership to TS 8eb3bf571c5a43e686ae4dc869c789f9
I20260709 13:14:27.058463 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:27.058856 10460 raft_consensus.cc:606] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Received request to transfer leadership to TS 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:27.061049 10460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "3884684625ea4fd58eae56cf80373794"
mode: GRACEFUL
new_leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:47614
I20260709 13:14:27.061453 10460 raft_consensus.cc:606] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Received request to transfer leadership to TS 9d248dbd1e694e3fb2ada1133206ab5a
I20260709 13:14:27.062402 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:27.062851 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:27.073905 10617 raft_consensus.cc:993] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.074438 10729 raft_consensus.cc:1081] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.076006 10385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46542
I20260709 13:14:27.076597 10385 raft_consensus.cc:493] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:27.077025 10385 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:27.083248 10385 raft_consensus.cc:515] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.085150 10385 leader_election.cc:290] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:27.085880 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:27.086541 10460 raft_consensus.cc:3055] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:27.086505 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a649d4e43914f32864c4d69759c75e1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:27.086849 10460 raft_consensus.cc:740] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:27.087157 10535 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:27.087654 10460 consensus_queue.cc:260] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.088474 10460 raft_consensus.cc:3060] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:27.092561 10535 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:27.093355 10460 raft_consensus.cc:2468] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 2.
I20260709 13:14:27.094084 10350 leader_election.cc:304] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:27.095515 10730 raft_consensus.cc:2804] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:27.096319 10730 raft_consensus.cc:697] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:27.097208 10730 consensus_queue.cc:237] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.104353 10279 catalog_manager.cc:5697] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 1 to 2, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.119679 10617 raft_consensus.cc:993] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.120064 10729 raft_consensus.cc:1081] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.121392 10385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46542
I20260709 13:14:27.121785 10385 raft_consensus.cc:493] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:27.122006 10385 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:27.125945 10385 raft_consensus.cc:515] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.127458 10385 leader_election.cc:290] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 5 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:27.128155 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:27.128424 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b999ae81d275456e856cc7df04961eb1" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:27.128717 10460 raft_consensus.cc:3055] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:27.128947 10535 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:27.129076 10460 raft_consensus.cc:740] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:27.129933 10460 consensus_queue.cc:260] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.130921 10460 raft_consensus.cc:3060] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:27.133606 10535 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 5.
I20260709 13:14:27.134600 10350 leader_election.cc:304] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:27.135465 10730 raft_consensus.cc:2804] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:27.135924 10730 raft_consensus.cc:697] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 5 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:27.135865 10460 raft_consensus.cc:2468] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 5.
I20260709 13:14:27.137346 10730 consensus_queue.cc:237] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.144014 10279 catalog_manager.cc:5697] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 4 to 5, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 5 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.490934 10535 raft_consensus.cc:1275] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:27.492226 10730 consensus_queue.cc:1048] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:27.502867 10460 raft_consensus.cc:1275] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:27.504585 10730 consensus_queue.cc:1048] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:27.535012 10729 raft_consensus.cc:993] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.535454 10729 raft_consensus.cc:1081] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Signalling peer 8eb3bf571c5a43e686ae4dc869c789f9 to start an election
I20260709 13:14:27.536775 10385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8"
dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
 from {username='slave'} at 127.0.0.1:46542
I20260709 13:14:27.537205 10385 raft_consensus.cc:493] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:27.537477 10385 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.542341 10385 raft_consensus.cc:515] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.544096 10385 leader_election.cc:290] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 4 election: Requested vote from peers 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061), 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3:42953)
I20260709 13:14:27.544756 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:27.545338 10460 raft_consensus.cc:3055] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:27.545436 10535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b702b1851a79469391352d39a2ea1ce8" candidate_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
I20260709 13:14:27.545665 10460 raft_consensus.cc:740] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:27.545964 10535 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.546401 10460 consensus_queue.cc:260] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.547497 10460 raft_consensus.cc:3060] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.552495 10535 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 4.
I20260709 13:14:27.553539 10460 raft_consensus.cc:2468] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb3bf571c5a43e686ae4dc869c789f9 in term 4.
I20260709 13:14:27.553603 10350 leader_election.cc:304] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:27.554461 10730 raft_consensus.cc:2804] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:27.554968 10730 raft_consensus.cc:697] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 LEADER]: Becoming Leader. State: Replica: 8eb3bf571c5a43e686ae4dc869c789f9, State: Running, Role: LEADER
I20260709 13:14:27.555742 10730 consensus_queue.cc:237] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.562541 10280 catalog_manager.cc:5697] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 reported cstate change: term changed from 3 to 4, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.575310 10617 raft_consensus.cc:993] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: : Instructing follower 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:27.575731 10729 raft_consensus.cc:1081] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Signalling peer 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:27.577055 10535 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7aef538eb944672a1755ead81eae573"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:33884
I20260709 13:14:27.577507 10535 raft_consensus.cc:493] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:27.577807 10535 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.582155 10535 raft_consensus.cc:515] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:27.583906 10535 leader_election.cc:290] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 4 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:27.584641 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:27.585006 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aef538eb944672a1755ead81eae573" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:27.585209 10385 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.585517 10460 raft_consensus.cc:3055] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:27.585774 10460 raft_consensus.cc:740] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:27.586346 10460 consensus_queue.cc:260] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:27.587155 10460 raft_consensus.cc:3060] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:27.589310 10385 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 4.
I20260709 13:14:27.590418 10500 leader_election.cc:304] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:27.591315 10709 raft_consensus.cc:2804] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:27.591490 10460 raft_consensus.cc:2468] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 4.
I20260709 13:14:27.592844 10709 raft_consensus.cc:697] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:27.593922 10709 consensus_queue.cc:237] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } }
I20260709 13:14:27.595259 10617 raft_consensus.cc:993] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: : Instructing follower 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:27.595726 10729 raft_consensus.cc:1081] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Signalling peer 9d248dbd1e694e3fb2ada1133206ab5a to start an election
I20260709 13:14:27.597352 10535 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da"
dest_uuid: "9d248dbd1e694e3fb2ada1133206ab5a"
 from {username='slave'} at 127.0.0.1:33884
I20260709 13:14:27.597922 10535 raft_consensus.cc:493] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:27.598311 10535 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:27.601969 10280 catalog_manager.cc:5697] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 3 to 4, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 4 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.604362 10535 raft_consensus.cc:515] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.605865 10535 leader_election.cc:290] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 3 election: Requested vote from peers 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701), 3884684625ea4fd58eae56cf80373794 (127.10.3.2:44061)
I20260709 13:14:27.606760 10385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eb3bf571c5a43e686ae4dc869c789f9"
I20260709 13:14:27.606920 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0377ed34d4a497683fd25ab2179b8da" candidate_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3884684625ea4fd58eae56cf80373794"
I20260709 13:14:27.607333 10385 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:27.607496 10460 raft_consensus.cc:3055] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:27.607837 10460 raft_consensus.cc:740] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3884684625ea4fd58eae56cf80373794, State: Running, Role: LEADER
I20260709 13:14:27.608537 10460 consensus_queue.cc:260] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.609351 10460 raft_consensus.cc:3060] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:27.613056 10385 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 3.
I20260709 13:14:27.614096 10460 raft_consensus.cc:2468] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d248dbd1e694e3fb2ada1133206ab5a in term 3.
I20260709 13:14:27.614007 10500 leader_election.cc:304] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eb3bf571c5a43e686ae4dc869c789f9, 9d248dbd1e694e3fb2ada1133206ab5a; no voters: 
I20260709 13:14:27.614965 10709 raft_consensus.cc:2804] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:27.615440 10709 raft_consensus.cc:697] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Becoming Leader. State: Replica: 9d248dbd1e694e3fb2ada1133206ab5a, State: Running, Role: LEADER
I20260709 13:14:27.616163 10709 consensus_queue.cc:237] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } }
I20260709 13:14:27.622745 10280 catalog_manager.cc:5697] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a reported cstate change: term changed from 2 to 3, leader changed from 3884684625ea4fd58eae56cf80373794 (127.10.3.2) to 9d248dbd1e694e3fb2ada1133206ab5a (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb3bf571c5a43e686ae4dc869c789f9" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:27.636953 10460 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 5 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:27.638068 10730 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:27.647634 10535 raft_consensus.cc:1275] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 5 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:27.648908 10740 consensus_queue.cc:1048] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:27.994221 10535 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:27.995635 10731 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d248dbd1e694e3fb2ada1133206ab5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:28.004910 10460 raft_consensus.cc:1275] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Refusing update from remote peer 8eb3bf571c5a43e686ae4dc869c789f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:28.007184 10740 consensus_queue.cc:1048] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:28.063733 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:28.068972 10252 tablet_server.cc:179] TabletServer@127.10.3.1:0 shutting down...
W20260709 13:14:28.101539 10500 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:28.102069 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:28.102900 10252 tablet_replica.cc:333] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.103739 10252 raft_consensus.cc:2243] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.104300 10252 raft_consensus.cc:2272] T d0377ed34d4a497683fd25ab2179b8da P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.106868 10252 tablet_replica.cc:333] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.107657 10252 raft_consensus.cc:2243] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.108628 10252 raft_consensus.cc:2272] T b999ae81d275456e856cc7df04961eb1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:28.110723 10500 consensus_peers.cc:597] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a -> Peer 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701): Couldn't send request to peer 8eb3bf571c5a43e686ae4dc869c789f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:28.111423 10252 tablet_replica.cc:333] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.111903 10252 raft_consensus.cc:2243] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.112263 10252 raft_consensus.cc:2272] T b764787ab3314c65a5a309233c95f29b P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.114008 10252 tablet_replica.cc:333] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.114512 10252 raft_consensus.cc:2243] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.114912 10252 raft_consensus.cc:2272] T ac214c4415ba45b9a122b5b5b14cf0e2 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.115643 10460 raft_consensus.cc:1275] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:28.116982 10252 tablet_replica.cc:333] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.117759 10252 raft_consensus.cc:2243] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.117532 10709 consensus_queue.cc:1048] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:28.118302 10252 raft_consensus.cc:2272] T d7aef538eb944672a1755ead81eae573 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.119407 10460 raft_consensus.cc:1275] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Refusing update from remote peer 9d248dbd1e694e3fb2ada1133206ab5a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:28.121351 10252 tablet_replica.cc:333] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.122186 10252 raft_consensus.cc:2243] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.122108 10709 consensus_queue.cc:1048] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3884684625ea4fd58eae56cf80373794" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:28.123236 10252 raft_consensus.cc:2272] T 8a649d4e43914f32864c4d69759c75e1 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.126053 10252 tablet_replica.cc:333] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.126845 10252 raft_consensus.cc:2243] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.127462 10252 raft_consensus.cc:2272] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 8eb3bf571c5a43e686ae4dc869c789f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.129899 10252 tablet_replica.cc:333] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.130541 10252 raft_consensus.cc:2243] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.131863 10252 raft_consensus.cc:2272] T b702b1851a79469391352d39a2ea1ce8 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.134898 10252 tablet_replica.cc:333] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9: stopping tablet replica
I20260709 13:14:28.135914 10252 raft_consensus.cc:2243] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.136646 10252 raft_consensus.cc:2272] T 61c258b6385e4d63a206c4c011086ad9 P 8eb3bf571c5a43e686ae4dc869c789f9 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:28.138424 10500 consensus_peers.cc:597] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a -> Peer 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701): Couldn't send request to peer 8eb3bf571c5a43e686ae4dc869c789f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:28.240819 10252 tablet_server.cc:196] TabletServer@127.10.3.1:0 shutdown complete.
W20260709 13:14:28.242434 10426 consensus_peers.cc:597] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 -> Peer 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701): Couldn't send request to peer 8eb3bf571c5a43e686ae4dc869c789f9. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:28.244886 10426 consensus_peers.cc:597] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 -> Peer 8eb3bf571c5a43e686ae4dc869c789f9 (127.10.3.1:42701): Couldn't send request to peer 8eb3bf571c5a43e686ae4dc869c789f9. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:42701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:28.258708 10252 tablet_server.cc:179] TabletServer@127.10.3.2:0 shutting down...
I20260709 13:14:28.283452 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:28.284042 10252 tablet_replica.cc:333] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.284610 10252 raft_consensus.cc:2243] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.285125 10252 raft_consensus.cc:2272] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.286834 10252 tablet_replica.cc:333] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.287355 10252 raft_consensus.cc:2243] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.287756 10252 raft_consensus.cc:2272] T 8a649d4e43914f32864c4d69759c75e1 P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.289659 10252 tablet_replica.cc:333] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.290155 10252 raft_consensus.cc:2243] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.290542 10252 raft_consensus.cc:2272] T d0377ed34d4a497683fd25ab2179b8da P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.292656 10252 tablet_replica.cc:333] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.293133 10252 raft_consensus.cc:2243] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.293542 10252 raft_consensus.cc:2272] T b999ae81d275456e856cc7df04961eb1 P 3884684625ea4fd58eae56cf80373794 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.295857 10252 tablet_replica.cc:333] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.296335 10252 raft_consensus.cc:2243] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.296722 10252 raft_consensus.cc:2272] T b702b1851a79469391352d39a2ea1ce8 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.298724 10252 tablet_replica.cc:333] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.299228 10252 raft_consensus.cc:2243] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.299922 10252 raft_consensus.cc:2272] T 61c258b6385e4d63a206c4c011086ad9 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.301466 10252 tablet_replica.cc:333] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.301923 10252 raft_consensus.cc:2243] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.302291 10252 raft_consensus.cc:2272] T d7aef538eb944672a1755ead81eae573 P 3884684625ea4fd58eae56cf80373794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.304018 10252 tablet_replica.cc:333] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.304451 10252 raft_consensus.cc:2243] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.305181 10252 raft_consensus.cc:2272] T b764787ab3314c65a5a309233c95f29b P 3884684625ea4fd58eae56cf80373794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.306667 10252 tablet_replica.cc:333] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794: stopping tablet replica
I20260709 13:14:28.307149 10252 raft_consensus.cc:2243] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.307816 10252 raft_consensus.cc:2272] T ac214c4415ba45b9a122b5b5b14cf0e2 P 3884684625ea4fd58eae56cf80373794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.335111 10252 tablet_server.cc:196] TabletServer@127.10.3.2:0 shutdown complete.
I20260709 13:14:28.350961 10252 tablet_server.cc:179] TabletServer@127.10.3.3:0 shutting down...
I20260709 13:14:28.375953 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:28.376649 10252 tablet_replica.cc:333] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.377502 10252 raft_consensus.cc:2243] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.378096 10252 raft_consensus.cc:2272] T 8a649d4e43914f32864c4d69759c75e1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.380154 10252 tablet_replica.cc:333] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.380585 10252 raft_consensus.cc:2243] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.381230 10252 raft_consensus.cc:2272] T 6c5dbfb73bbc469a8c28e69d559c4a3a P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.382810 10252 tablet_replica.cc:333] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.383347 10252 raft_consensus.cc:2243] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.383766 10252 raft_consensus.cc:2272] T b764787ab3314c65a5a309233c95f29b P 9d248dbd1e694e3fb2ada1133206ab5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.385281 10252 tablet_replica.cc:333] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.385757 10252 raft_consensus.cc:2243] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.386154 10252 raft_consensus.cc:2272] T ac214c4415ba45b9a122b5b5b14cf0e2 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.387749 10252 tablet_replica.cc:333] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.388236 10252 raft_consensus.cc:2243] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.388931 10252 raft_consensus.cc:2272] T d7aef538eb944672a1755ead81eae573 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.390677 10252 tablet_replica.cc:333] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.391232 10252 raft_consensus.cc:2243] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.391916 10252 raft_consensus.cc:2272] T d0377ed34d4a497683fd25ab2179b8da P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.393561 10252 tablet_replica.cc:333] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.394035 10252 raft_consensus.cc:2243] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.394424 10252 raft_consensus.cc:2272] T b999ae81d275456e856cc7df04961eb1 P 9d248dbd1e694e3fb2ada1133206ab5a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.396457 10252 tablet_replica.cc:333] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.396924 10252 raft_consensus.cc:2243] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.397333 10252 raft_consensus.cc:2272] T b702b1851a79469391352d39a2ea1ce8 P 9d248dbd1e694e3fb2ada1133206ab5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.399185 10252 tablet_replica.cc:333] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a: stopping tablet replica
I20260709 13:14:28.399613 10252 raft_consensus.cc:2243] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:28.400027 10252 raft_consensus.cc:2272] T 61c258b6385e4d63a206c4c011086ad9 P 9d248dbd1e694e3fb2ada1133206ab5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.426139 10252 tablet_server.cc:196] TabletServer@127.10.3.3:0 shutdown complete.
I20260709 13:14:28.441622 10252 master.cc:562] Master@127.10.3.62:44419 shutting down...
I20260709 13:14:28.620138 10252 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:28.620637 10252 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:28.620909 10252 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0090722fbc194c6e919a826e4d6020e6: stopping tablet replica
I20260709 13:14:28.642500 10252 master.cc:584] Master@127.10.3.62:44419 shutdown complete.
3884684625ea4fd58eae56cf80373794  4
8eb3bf571c5a43e686ae4dc869c789f9  4
9d248dbd1e694e3fb2ada1133206ab5a  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12838 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:14:28.683522 10252 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.3.62:32873
I20260709 13:14:28.684579 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:28.689838 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:28.690135 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:28.691725 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:28.691797 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:28.692755 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:28.692963 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:28.693128 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602868693111 us; error 0 us; skew 500 ppm
I20260709 13:14:28.693677 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:28.696425 10252 webserver.cc:533] Webserver started at http://127.10.3.62:35325/ using document root <none> and password file <none>
I20260709 13:14:28.696952 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:28.697165 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:28.697440 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:28.698529 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/master-0-root/instance:
uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:28.702745 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:14:28.705883 10767 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:28.706873 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:28.707158 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/master-0-root
uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:28.707391 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:28.731106 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:28.732275 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:28.768954 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.62:32873
I20260709 13:14:28.769047 10818 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.62:32873 every 8 connection(s)
I20260709 13:14:28.773288 10819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:28.782233 10819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9: Bootstrap starting.
I20260709 13:14:28.786587 10819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:28.790624 10819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9: No bootstrap required, opened a new log
I20260709 13:14:28.792595 10819 raft_consensus.cc:359] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER }
I20260709 13:14:28.792963 10819 raft_consensus.cc:385] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:28.793134 10819 raft_consensus.cc:740] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564ba8ab7cbf4e1191d4a095bf9bc9a9, State: Initialized, Role: FOLLOWER
I20260709 13:14:28.793772 10819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER }
I20260709 13:14:28.794178 10819 raft_consensus.cc:399] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:28.794360 10819 raft_consensus.cc:493] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:28.794561 10819 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:28.798545 10819 raft_consensus.cc:515] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER }
I20260709 13:14:28.799084 10819 leader_election.cc:304] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 564ba8ab7cbf4e1191d4a095bf9bc9a9; no voters: 
I20260709 13:14:28.800113 10819 leader_election.cc:290] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:28.800549 10822 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:28.801877 10822 raft_consensus.cc:697] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 564ba8ab7cbf4e1191d4a095bf9bc9a9, State: Running, Role: LEADER
I20260709 13:14:28.802629 10822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER }
I20260709 13:14:28.803035 10819 sys_catalog.cc:565] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:28.807880 10824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564ba8ab7cbf4e1191d4a095bf9bc9a9. Latest consensus state: current_term: 1 leader_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER } }
I20260709 13:14:28.807989 10823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ba8ab7cbf4e1191d4a095bf9bc9a9" member_type: VOTER } }
I20260709 13:14:28.808841 10823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:28.808952 10824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:28.811725 10830 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:28.818439 10830 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:28.819474 10252 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:28.827265 10830 catalog_manager.cc:1383] Generated new cluster ID: 5b82860bb9e74b048fa1d6afafc5b6f8
I20260709 13:14:28.827540 10830 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:28.849404 10830 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:28.850656 10830 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:28.866187 10830 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9: Generated new TSK 0
I20260709 13:14:28.866745 10830 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:28.886222 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:28.891860 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:28.892830 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:28.895596 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:28.895617 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:28.896615 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:28.896835 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:28.897009 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602868896992 us; error 0 us; skew 500 ppm
I20260709 13:14:28.897528 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:28.899812 10252 webserver.cc:533] Webserver started at http://127.10.3.1:42067/ using document root <none> and password file <none>
I20260709 13:14:28.900300 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:28.900501 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:28.900769 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:28.901813 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-0-root/instance:
uuid: "6fedf8972c484b14a503b9d9b498a317"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:28.906458 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:14:28.909835 10848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:28.910591 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:14:28.910887 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
uuid: "6fedf8972c484b14a503b9d9b498a317"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:28.911213 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:28.922747 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:28.924113 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:28.925536 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:28.927735 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:28.927950 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:28.928196 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:28.928385 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:28.970891 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.1:41007
I20260709 13:14:28.971056 10910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.1:41007 every 8 connection(s)
I20260709 13:14:28.975751 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:28.983664 10915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:28.985146 10916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:28.988230 10918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:28.989436 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:28.990278 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:28.990492 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:28.990724 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602868990676 us; error 0 us; skew 500 ppm
I20260709 13:14:28.991293 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:28.993582 10252 webserver.cc:533] Webserver started at http://127.10.3.2:36107/ using document root <none> and password file <none>
I20260709 13:14:28.994102 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:28.994350 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:28.994690 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:28.996209 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-1-root/instance:
uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:28.997455 10911 heartbeater.cc:344] Connected to a master server at 127.10.3.62:32873
I20260709 13:14:28.997802 10911 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:28.998483 10911 heartbeater.cc:507] Master 127.10.3.62:32873 requested a full tablet report, sending...
I20260709 13:14:29.000594 10784 ts_manager.cc:194] Registered new tserver with Master: 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007)
I20260709 13:14:29.001235 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:14:29.002404 10784 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35122
I20260709 13:14:29.004861 10923 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.005596 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:14:29.005842 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
format_stamp: "Formatted at 2026-07-09 13:14:28 on dist-test-slave-gvjr"
I20260709 13:14:29.006074 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:29.036650 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:29.037825 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:29.039213 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:29.041280 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:29.041446 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.041738 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:29.041894 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.081151 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.2:38059
I20260709 13:14:29.081241 10985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.2:38059 every 8 connection(s)
I20260709 13:14:29.086103 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:29.093976 10989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:29.094643 10990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:29.097956 10992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:29.099121 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:29.100153 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:14:29.100420 10986 heartbeater.cc:344] Connected to a master server at 127.10.3.62:32873
W20260709 13:14:29.100436 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:29.100862 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602869100837 us; error 0 us; skew 500 ppm
I20260709 13:14:29.100893 10986 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:29.101581 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:29.101768 10986 heartbeater.cc:507] Master 127.10.3.62:32873 requested a full tablet report, sending...
I20260709 13:14:29.104030 10784 ts_manager.cc:194] Registered new tserver with Master: 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:29.104410 10252 webserver.cc:533] Webserver started at http://127.10.3.3:34543/ using document root <none> and password file <none>
I20260709 13:14:29.105000 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:29.105194 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:29.105476 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:29.105736 10784 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35136
I20260709 13:14:29.106841 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-2-root/instance:
uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
format_stamp: "Formatted at 2026-07-09 13:14:29 on dist-test-slave-gvjr"
I20260709 13:14:29.111591 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:29.115109 10997 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.115873 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:29.116196 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
format_stamp: "Formatted at 2026-07-09 13:14:29 on dist-test-slave-gvjr"
I20260709 13:14:29.116480 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602855680335-10252-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:29.132685 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:29.133942 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:29.135411 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:29.137488 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:29.137704 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.137948 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:29.138119 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:29.176558 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.3:46831
I20260709 13:14:29.176654 11059 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.3:46831 every 8 connection(s)
I20260709 13:14:29.189766 11060 heartbeater.cc:344] Connected to a master server at 127.10.3.62:32873
I20260709 13:14:29.190073 11060 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:29.190690 11060 heartbeater.cc:507] Master 127.10.3.62:32873 requested a full tablet report, sending...
I20260709 13:14:29.192540 10784 ts_manager.cc:194] Registered new tserver with Master: 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.193161 10252 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013457337s
I20260709 13:14:29.193876 10784 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35146
I20260709 13:14:29.218092 10784 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35148:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:29.221001 10784 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:29.296602 10876 tablet_service.cc:1511] Processing CreateTablet for tablet 6febcca359294eeba020eb05b75c5eee (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:29.297222 10874 tablet_service.cc:1511] Processing CreateTablet for tablet 0908a580b50644668401a9ecb02d481a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:29.297541 10873 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab2b71d68a940deba12d61b570cec68 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:29.298118 10876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6febcca359294eeba020eb05b75c5eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.298743 10874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0908a580b50644668401a9ecb02d481a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.299405 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab2b71d68a940deba12d61b570cec68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.300034 10875 tablet_service.cc:1511] Processing CreateTablet for tablet 6a682a9d136e4a1d8549766f16b0dcf7 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:29.301326 10875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a682a9d136e4a1d8549766f16b0dcf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.301828 10872 tablet_service.cc:1511] Processing CreateTablet for tablet 47309bbe949949e7823a02e151e39441 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:29.303086 10872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47309bbe949949e7823a02e151e39441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.305826 10871 tablet_service.cc:1511] Processing CreateTablet for tablet 474f8e3b3e29485a8bc3ec260fa7c69d (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:29.307143 10871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474f8e3b3e29485a8bc3ec260fa7c69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.309827 10870 tablet_service.cc:1511] Processing CreateTablet for tablet 307db797d62949eba0c77576ca9ae5ce (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:29.311144 10870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307db797d62949eba0c77576ca9ae5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.313767 10869 tablet_service.cc:1511] Processing CreateTablet for tablet c5be0ff321734d5ba0eb8782c3d0fb0c (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:29.315265 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5be0ff321734d5ba0eb8782c3d0fb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.322057 10868 tablet_service.cc:1511] Processing CreateTablet for tablet 913b6d1c044f46c28c06b34cb355fd48 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:29.323463 10868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 913b6d1c044f46c28c06b34cb355fd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.329365 10867 tablet_service.cc:1511] Processing CreateTablet for tablet f10cbaa44a3e4e0c99aa510e14fab02a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:29.330780 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f10cbaa44a3e4e0c99aa510e14fab02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.341936 11025 tablet_service.cc:1511] Processing CreateTablet for tablet 6febcca359294eeba020eb05b75c5eee (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:29.344156 11024 tablet_service.cc:1511] Processing CreateTablet for tablet 6a682a9d136e4a1d8549766f16b0dcf7 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:29.345450 11024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a682a9d136e4a1d8549766f16b0dcf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.348189 11023 tablet_service.cc:1511] Processing CreateTablet for tablet 0908a580b50644668401a9ecb02d481a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:29.349463 11023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0908a580b50644668401a9ecb02d481a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.352218 11022 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab2b71d68a940deba12d61b570cec68 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:29.353509 11022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab2b71d68a940deba12d61b570cec68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.356204 11021 tablet_service.cc:1511] Processing CreateTablet for tablet 47309bbe949949e7823a02e151e39441 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:29.357508 11021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47309bbe949949e7823a02e151e39441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.360244 11020 tablet_service.cc:1511] Processing CreateTablet for tablet 474f8e3b3e29485a8bc3ec260fa7c69d (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:29.361555 11020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474f8e3b3e29485a8bc3ec260fa7c69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.364064 11019 tablet_service.cc:1511] Processing CreateTablet for tablet 307db797d62949eba0c77576ca9ae5ce (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:29.365387 11019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307db797d62949eba0c77576ca9ae5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.368175 11018 tablet_service.cc:1511] Processing CreateTablet for tablet c5be0ff321734d5ba0eb8782c3d0fb0c (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:29.369513 11018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5be0ff321734d5ba0eb8782c3d0fb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.371280 10943 tablet_service.cc:1511] Processing CreateTablet for tablet 913b6d1c044f46c28c06b34cb355fd48 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:29.371585 10951 tablet_service.cc:1511] Processing CreateTablet for tablet 6febcca359294eeba020eb05b75c5eee (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:29.372594 10943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 913b6d1c044f46c28c06b34cb355fd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.373296 10950 tablet_service.cc:1511] Processing CreateTablet for tablet 6a682a9d136e4a1d8549766f16b0dcf7 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:29.375272 10949 tablet_service.cc:1511] Processing CreateTablet for tablet 0908a580b50644668401a9ecb02d481a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:29.376150 10948 tablet_service.cc:1511] Processing CreateTablet for tablet 7ab2b71d68a940deba12d61b570cec68 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:29.377990 10947 tablet_service.cc:1511] Processing CreateTablet for tablet 47309bbe949949e7823a02e151e39441 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:29.383561 10945 tablet_service.cc:1511] Processing CreateTablet for tablet 307db797d62949eba0c77576ca9ae5ce (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:29.387992 11017 tablet_service.cc:1511] Processing CreateTablet for tablet 913b6d1c044f46c28c06b34cb355fd48 (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:29.390926 10944 tablet_service.cc:1511] Processing CreateTablet for tablet c5be0ff321734d5ba0eb8782c3d0fb0c (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:29.382663 10946 tablet_service.cc:1511] Processing CreateTablet for tablet 474f8e3b3e29485a8bc3ec260fa7c69d (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:29.392570 10942 tablet_service.cc:1511] Processing CreateTablet for tablet f10cbaa44a3e4e0c99aa510e14fab02a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:29.393344 11025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6febcca359294eeba020eb05b75c5eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.394060 11017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 913b6d1c044f46c28c06b34cb355fd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.396675 11016 tablet_service.cc:1511] Processing CreateTablet for tablet f10cbaa44a3e4e0c99aa510e14fab02a (DEFAULT_TABLE table=test-workload [id=58e6cf733743481095a1dc51e095f1d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:29.397491 10946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474f8e3b3e29485a8bc3ec260fa7c69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.397939 11016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f10cbaa44a3e4e0c99aa510e14fab02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.403601 10949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0908a580b50644668401a9ecb02d481a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.407619 10948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ab2b71d68a940deba12d61b570cec68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.411513 10947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47309bbe949949e7823a02e151e39441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.412403 10945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307db797d62949eba0c77576ca9ae5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.423885 11075 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.432720 10950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a682a9d136e4a1d8549766f16b0dcf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.433460 11076 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.434814 10951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6febcca359294eeba020eb05b75c5eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.438215 10944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5be0ff321734d5ba0eb8782c3d0fb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.443276 10942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f10cbaa44a3e4e0c99aa510e14fab02a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:29.470711 11076 tablet_bootstrap.cc:654] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.482018 11075 tablet_bootstrap.cc:654] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.522053 11075 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.522619 11075 ts_tablet_manager.cc:1403] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.005s
I20260709 13:14:29.526197 11077 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.531217 11077 tablet_bootstrap.cc:654] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.540539 11075 raft_consensus.cc:359] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.542451 11075 raft_consensus.cc:385] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.542728 11075 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.543419 11075 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.548414 10911 heartbeater.cc:499] Master 127.10.3.62:32873 was elected leader, sending a full tablet report...
I20260709 13:14:29.548903 11075 ts_tablet_manager.cc:1434] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 13:14:29.549249 11076 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.549670 11076 ts_tablet_manager.cc:1403] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.117s	user 0.020s	sys 0.017s
I20260709 13:14:29.549887 11075 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.550635 11077 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.551084 11077 ts_tablet_manager.cc:1403] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.012s
I20260709 13:14:29.552500 11076 raft_consensus.cc:359] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.553355 11076 raft_consensus.cc:385] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.553803 11076 raft_consensus.cc:740] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.553880 11077 raft_consensus.cc:359] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.554715 11077 raft_consensus.cc:385] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.555115 11077 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.554832 11076 consensus_queue.cc:260] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.555946 11077 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.556450 11075 tablet_bootstrap.cc:654] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.557798 11060 heartbeater.cc:499] Master 127.10.3.62:32873 was elected leader, sending a full tablet report...
I20260709 13:14:29.558735 10986 heartbeater.cc:499] Master 127.10.3.62:32873 was elected leader, sending a full tablet report...
I20260709 13:14:29.562851 11077 ts_tablet_manager.cc:1434] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:14:29.563035 11076 ts_tablet_manager.cc:1434] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:14:29.563860 11077 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.564113 11076 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.570959 11076 tablet_bootstrap.cc:654] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.571038 11077 tablet_bootstrap.cc:654] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.573114 11075 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.573668 11075 ts_tablet_manager.cc:1403] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:14:29.577431 11075 raft_consensus.cc:359] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.578436 11075 raft_consensus.cc:385] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.579011 11075 raft_consensus.cc:740] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.580143 11075 consensus_queue.cc:260] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.583386 11075 ts_tablet_manager.cc:1434] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:14:29.584643 11075 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.593182 11075 tablet_bootstrap.cc:654] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.596422 11076 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.597129 11076 ts_tablet_manager.cc:1403] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.020s
I20260709 13:14:29.601011 11076 raft_consensus.cc:359] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.602169 11076 raft_consensus.cc:385] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.602618 11076 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.603890 11076 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.607156 11076 ts_tablet_manager.cc:1434] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:14:29.608551 11076 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.617690 11076 tablet_bootstrap.cc:654] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.637898 11077 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.638471 11077 ts_tablet_manager.cc:1403] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.055s
I20260709 13:14:29.640594 11075 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.641010 11075 ts_tablet_manager.cc:1403] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.057s	user 0.032s	sys 0.021s
I20260709 13:14:29.640920 11077 raft_consensus.cc:359] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.641541 11077 raft_consensus.cc:385] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.641814 11077 raft_consensus.cc:740] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.642462 11077 consensus_queue.cc:260] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.643481 11075 raft_consensus.cc:359] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.644253 11075 raft_consensus.cc:385] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.644387 11077 ts_tablet_manager.cc:1434] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:14:29.644606 11075 raft_consensus.cc:740] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.645241 11076 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.645445 11077 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.645742 11076 ts_tablet_manager.cc:1403] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.007s
I20260709 13:14:29.645619 11075 consensus_queue.cc:260] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.647902 11075 ts_tablet_manager.cc:1434] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:14:29.648882 11075 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.651201 11081 raft_consensus.cc:493] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.651672 11081 raft_consensus.cc:515] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.652138 11077 tablet_bootstrap.cc:654] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.654320 11081 leader_election.cc:290] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.655726 11075 tablet_bootstrap.cc:654] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.658002 11076 raft_consensus.cc:359] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.658962 11076 raft_consensus.cc:385] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.659449 11076 raft_consensus.cc:740] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.660584 11076 consensus_queue.cc:260] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.663576 11076 ts_tablet_manager.cc:1434] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.005s
I20260709 13:14:29.664800 11076 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.670974 11077 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.671533 11077 ts_tablet_manager.cc:1403] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.012s
I20260709 13:14:29.671856 11076 tablet_bootstrap.cc:654] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.674628 11077 raft_consensus.cc:359] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.675711 11077 raft_consensus.cc:385] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.676158 11077 raft_consensus.cc:740] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.677335 11077 consensus_queue.cc:260] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.682824 11077 ts_tablet_manager.cc:1434] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:29.684240 11077 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.692869 11077 tablet_bootstrap.cc:654] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.697525 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:29.698247 11075 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.698706 11075 ts_tablet_manager.cc:1403] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 13:14:29.698523 11035 raft_consensus.cc:2468] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 0.
I20260709 13:14:29.700193 10850 leader_election.cc:304] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:29.701084 11081 raft_consensus.cc:2804] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:29.701413 11081 raft_consensus.cc:493] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.701694 11081 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.703588 11076 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.704142 11076 ts_tablet_manager.cc:1403] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.012s
I20260709 13:14:29.705124 11077 tablet_bootstrap.cc:492] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.705063 11075 raft_consensus.cc:359] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.705722 11077 ts_tablet_manager.cc:1403] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.001s
I20260709 13:14:29.705886 11075 raft_consensus.cc:385] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.706235 11075 raft_consensus.cc:740] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.707216 11076 raft_consensus.cc:359] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.708026 11076 raft_consensus.cc:385] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.708380 11076 raft_consensus.cc:740] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.708937 11077 raft_consensus.cc:359] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.708799 11075 consensus_queue.cc:260] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.709395 11076 consensus_queue.cc:260] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.709841 11077 raft_consensus.cc:385] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.710559 11077 raft_consensus.cc:740] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.711788 11077 consensus_queue.cc:260] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.712524 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:29.716214 11077 ts_tablet_manager.cc:1434] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:29.714190 11081 raft_consensus.cc:515] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.717238 11077 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
W20260709 13:14:29.719552 10852 leader_election.cc:343] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.721146 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:29.721757 11035 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.722271 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:29.723819 11077 tablet_bootstrap.cc:654] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:29.724331 10852 leader_election.cc:343] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.729998 11076 ts_tablet_manager.cc:1434] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:14:29.731632 11076 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.743301 11035 raft_consensus.cc:2468] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 1.
I20260709 13:14:29.745551 10850 leader_election.cc:304] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:29.744028 11081 leader_election.cc:290] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.747910 11076 tablet_bootstrap.cc:654] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.748785 11081 raft_consensus.cc:2804] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:29.756525 11081 raft_consensus.cc:697] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:29.757562 11081 consensus_queue.cc:237] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.762392 11076 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.762933 11076 ts_tablet_manager.cc:1403] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.006s
I20260709 13:14:29.763432 11075 ts_tablet_manager.cc:1434] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.064s	user 0.013s	sys 0.045s
I20260709 13:14:29.764585 11075 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.766551 11083 raft_consensus.cc:493] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.766840 11076 raft_consensus.cc:359] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.767680 11076 raft_consensus.cc:385] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.767884 11077 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.768007 11076 raft_consensus.cc:740] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.768375 11077 ts_tablet_manager.cc:1403] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.026s
I20260709 13:14:29.768926 11076 consensus_queue.cc:260] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.771070 11075 tablet_bootstrap.cc:654] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.771237 11077 raft_consensus.cc:359] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.771426 11083 raft_consensus.cc:515] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.772012 11077 raft_consensus.cc:385] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.772306 11077 raft_consensus.cc:740] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.773144 11076 ts_tablet_manager.cc:1434] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:29.773135 11077 consensus_queue.cc:260] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.774003 11083 leader_election.cc:290] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.774405 11076 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.782186 11077 ts_tablet_manager.cc:1434] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.007s
I20260709 13:14:29.784397 11077 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.770205 10782 catalog_manager.cc:5697] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:29.791887 11077 tablet_bootstrap.cc:654] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.804620 11076 tablet_bootstrap.cc:654] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:29.814806 10832 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:29.821244 11076 tablet_bootstrap.cc:492] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.821976 11076 ts_tablet_manager.cc:1403] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.009s
I20260709 13:14:29.827103 11076 raft_consensus.cc:359] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.828747 11076 raft_consensus.cc:385] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.829169 11076 raft_consensus.cc:740] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.830271 11076 consensus_queue.cc:260] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.834437 11077 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.834976 11077 ts_tablet_manager.cc:1403] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.033s
I20260709 13:14:29.838565 11077 raft_consensus.cc:359] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.838701 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.839450 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:29.840072 11075 tablet_bootstrap.cc:492] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.840551 11075 ts_tablet_manager.cc:1403] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.076s	user 0.011s	sys 0.023s
W20260709 13:14:29.840484 10926 leader_election.cc:343] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.841231 11077 raft_consensus.cc:385] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:29.841137 10925 leader_election.cc:343] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.841847 10925 leader_election.cc:304] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08; no voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:29.842640 11083 raft_consensus.cc:2749] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:29.843308 11076 ts_tablet_manager.cc:1434] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.006s
I20260709 13:14:29.843696 11077 raft_consensus.cc:740] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.843482 11075 raft_consensus.cc:359] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.844434 11076 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.844524 11077 consensus_queue.cc:260] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.847249 11077 ts_tablet_manager.cc:1434] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:14:29.844461 11075 raft_consensus.cc:385] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.848376 11077 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.848528 11075 raft_consensus.cc:740] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.849331 11075 consensus_queue.cc:260] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.851223 11076 tablet_bootstrap.cc:654] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.851598 11075 ts_tablet_manager.cc:1434] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:14:29.852624 11075 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.855508 11077 tablet_bootstrap.cc:654] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.859328 11075 tablet_bootstrap.cc:654] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.862620 11076 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.863173 11076 ts_tablet_manager.cc:1403] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.019s	user 0.003s	sys 0.015s
I20260709 13:14:29.866470 11076 raft_consensus.cc:359] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.867235 11076 raft_consensus.cc:385] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.867588 11076 raft_consensus.cc:740] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.868325 11076 consensus_queue.cc:260] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.870884 11076 ts_tablet_manager.cc:1434] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:14:29.871094 11082 raft_consensus.cc:493] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.872421 11082 raft_consensus.cc:515] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.872964 11076 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.876960 11077 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.877679 11077 ts_tablet_manager.cc:1403] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.016s
I20260709 13:14:29.879321 11076 tablet_bootstrap.cc:654] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.880788 11082 leader_election.cc:290] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:29.881603 11077 raft_consensus.cc:359] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.882632 11077 raft_consensus.cc:385] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.883085 11077 raft_consensus.cc:740] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.884254 11077 consensus_queue.cc:260] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.887488 11077 ts_tablet_manager.cc:1434] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:14:29.888453 11077 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.897125 11077 tablet_bootstrap.cc:654] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.900481 11076 tablet_bootstrap.cc:492] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:29.901007 11076 ts_tablet_manager.cc:1403] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.008s
I20260709 13:14:29.903566 11076 raft_consensus.cc:359] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.906435 11076 raft_consensus.cc:385] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.909351 11076 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.910043 11077 tablet_bootstrap.cc:492] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:29.910723 11077 ts_tablet_manager.cc:1403] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:14:29.912346 11076 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.914304 11077 raft_consensus.cc:359] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.915129 11076 ts_tablet_manager.cc:1434] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:14:29.915290 11077 raft_consensus.cc:385] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.917155 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.918658 10886 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:29.919013 11076 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:29.919631 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:29.920451 11075 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:29.920231 11000 leader_election.cc:304] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:29.920228 10961 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:29.920874 11083 raft_consensus.cc:493] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.921001 11075 ts_tablet_manager.cc:1403] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.069s	user 0.029s	sys 0.037s
I20260709 13:14:29.921429 11083 raft_consensus.cc:515] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.920894 11082 raft_consensus.cc:493] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.922420 11082 raft_consensus.cc:515] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.922973 11077 raft_consensus.cc:740] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.923830 11083 leader_election.cc:290] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.923626 11077 consensus_queue.cc:260] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.923960 11075 raft_consensus.cc:359] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.924664 11075 raft_consensus.cc:385] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:29.924994 11075 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:29.925113 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:29.925582 11077 ts_tablet_manager.cc:1434] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:14:29.925411 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.925695 11035 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 0.
I20260709 13:14:29.925839 11075 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.926054 10886 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 0.
I20260709 13:14:29.927600 10926 leader_election.cc:304] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:29.928032 11075 ts_tablet_manager.cc:1434] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:29.928426 11101 raft_consensus.cc:2804] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:29.928823 11101 raft_consensus.cc:493] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.928966 11075 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:29.929162 11101 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.935638 11075 tablet_bootstrap.cc:654] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.936892 11101 raft_consensus.cc:515] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.940510 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:29.941761 10886 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.937318 11077 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:29.959656 10886 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 1.
I20260709 13:14:29.960975 10926 leader_election.cc:304] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:29.961541 11076 tablet_bootstrap.cc:654] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.961865 11083 raft_consensus.cc:2804] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:29.964318 11082 leader_election.cc:290] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:29.965296 11101 leader_election.cc:290] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.965003 11082 raft_consensus.cc:2804] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:29.966755 11082 raft_consensus.cc:493] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.967147 11116 raft_consensus.cc:493] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.967274 11082 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.967794 11116 raft_consensus.cc:515] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.969976 11077 tablet_bootstrap.cc:654] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:29.973905 11081 raft_consensus.cc:493] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.970021 11116 leader_election.cc:290] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
W20260709 13:14:29.970803 11000 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:41007, user_credentials={real_user=slave}} blocked reactor thread for 50760.1us
I20260709 13:14:29.974567 11081 raft_consensus.cc:515] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.977061 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.977237 11081 leader_election.cc:290] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.977408 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.977663 10886 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:29.980060 11116 raft_consensus.cc:493] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.979941 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:29.981618 10961 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 0.
W20260709 13:14:29.980849 11000 leader_election.cc:343] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.983160 11000 leader_election.cc:304] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:29.983177 10852 leader_election.cc:304] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:29.984642 11081 raft_consensus.cc:2804] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:29.985181 11081 raft_consensus.cc:493] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.985651 11081 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:29.992833 11101 raft_consensus.cc:493] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.993427 11101 raft_consensus.cc:515] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:29.995836 11101 leader_election.cc:290] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:29.996970 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:29.997902 11077 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
W20260709 13:14:29.998522 10926 leader_election.cc:343] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:29.998771 11077 ts_tablet_manager.cc:1403] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.007s
I20260709 13:14:30.002882 11077 raft_consensus.cc:359] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.008162 11077 raft_consensus.cc:385] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.009167 11077 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.010635 11119 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.011333 11119 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.011482 11077 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.016187 11077 ts_tablet_manager.cc:1434] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:14:30.014708 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.018971 10961 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:29.984660 11116 raft_consensus.cc:515] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.017985 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.020406 10960 raft_consensus.cc:2468] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:30.018275 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.021409 11035 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 0.
I20260709 13:14:30.022873 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:30.023128 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.026906 11034 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.027123 11035 raft_consensus.cc:2468] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 0.
I20260709 13:14:30.031956 11077 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Bootstrap starting.
I20260709 13:14:30.033686 10925 leader_election.cc:304] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.035516 11101 raft_consensus.cc:2804] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.036008 11101 raft_consensus.cc:493] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.036470 11101 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.047451 11083 raft_consensus.cc:697] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:30.048569 11034 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 1.
I20260709 13:14:30.049917 11001 leader_election.cc:304] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.048450 11083 consensus_queue.cc:237] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.021240 11117 raft_consensus.cc:2804] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.052728 11077 tablet_bootstrap.cc:654] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:30.051923 11117 raft_consensus.cc:493] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.056317 11117 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.057314 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:30.057991 10886 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:30.059312 11000 leader_election.cc:304] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.059800 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:30.061746 11116 leader_election.cc:290] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.062425 11116 raft_consensus.cc:2804] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.002578 11081 raft_consensus.cc:515] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.046547 11101 raft_consensus.cc:515] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.062853 11116 raft_consensus.cc:493] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.070067 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.071560 11116 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.071521 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.071374 11076 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:30.072036 10960 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.064070 11119 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.072623 11076 ts_tablet_manager.cc:1403] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.154s	user 0.032s	sys 0.028s
I20260709 13:14:30.070966 10961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.073989 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:30.075428 11076 raft_consensus.cc:359] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.078184 11076 raft_consensus.cc:385] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.078264 11034 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.078498 11076 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.074900 11082 raft_consensus.cc:515] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.079208 11076 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.084360 11076 ts_tablet_manager.cc:1434] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:30.087345 11076 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: Bootstrap starting.
I20260709 13:14:30.064975 11120 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.087929 11120 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.089103 10959 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:30.090422 11120 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.094182 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
W20260709 13:14:30.096211 10926 leader_election.cc:343] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.097409 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.097853 11076 tablet_bootstrap.cc:654] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:30.098029 11035 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 0.
I20260709 13:14:30.075940 11119 raft_consensus.cc:2804] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.101436 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.104362 10925 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
W20260709 13:14:30.104637 10926 leader_election.cc:343] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.105532 11120 raft_consensus.cc:2804] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.105968 11120 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.106392 11120 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.077551 11081 leader_election.cc:290] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.122290 11075 tablet_bootstrap.cc:492] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:30.122923 11075 ts_tablet_manager.cc:1403] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.194s	user 0.045s	sys 0.094s
W20260709 13:14:30.091236 11001 leader_election.cc:343] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.129789 11082 leader_election.cc:290] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.111699 11119 raft_consensus.cc:493] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.131404 11119 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.139083 10960 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 1.
I20260709 13:14:30.139885 11077 tablet_bootstrap.cc:492] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: No bootstrap required, opened a new log
I20260709 13:14:30.140372 11077 ts_tablet_manager.cc:1403] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent bootstrapping tablet: real 0.109s	user 0.025s	sys 0.020s
W20260709 13:14:30.135206 11001 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:38059, user_credentials={real_user=slave}} blocked reactor thread for 52586.9us
I20260709 13:14:30.139140 11075 raft_consensus.cc:359] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.141420 11075 raft_consensus.cc:385] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.141722 11075 raft_consensus.cc:740] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.142495 11001 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.143628 11082 raft_consensus.cc:2804] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.144033 11082 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.145117 11082 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.147147 11077 raft_consensus.cc:359] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.147760 11077 raft_consensus.cc:385] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.148012 11077 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.150151 11117 raft_consensus.cc:515] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.151335 11077 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.140959 11101 leader_election.cc:290] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.157532 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:30.167950 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.170950 11130 raft_consensus.cc:493] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:30.168632 11000 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:41007, user_credentials={real_user=slave}} blocked reactor thread for 109395us
I20260709 13:14:30.171698 10960 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.169437 11101 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.171460 11130 raft_consensus.cc:515] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.173738 11101 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.175657 11101 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.176350 11077 ts_tablet_manager.cc:1434] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.004s
I20260709 13:14:30.142446 11075 consensus_queue.cc:260] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.176301 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.176499 10852 leader_election.cc:304] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:30.180207 10960 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.181310 10886 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.181797 11032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.182283 11001 leader_election.cc:304] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.182590 11032 raft_consensus.cc:2393] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in current term 1: Already voted for candidate 6fedf8972c484b14a503b9d9b498a317 in this term.
I20260709 13:14:30.183108 11075 ts_tablet_manager.cc:1434] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.000s
I20260709 13:14:30.177372 11033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.183667 11033 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 0.
I20260709 13:14:30.183856 11124 raft_consensus.cc:2804] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.184736 10925 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.185004 11081 raft_consensus.cc:2804] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.185531 11081 raft_consensus.cc:697] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:30.177111 11082 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.180440 11130 leader_election.cc:290] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.180174 11116 raft_consensus.cc:515] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.187495 11081 consensus_queue.cc:237] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.190115 11116 leader_election.cc:290] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
W20260709 13:14:30.177101 11000 leader_election.cc:343] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.187377 11101 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.188102 11075 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:30.192651 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.193379 10885 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.162297 11034 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 1.
I20260709 13:14:30.196702 11035 raft_consensus.cc:2393] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in current term 1: Already voted for candidate 7a55dd39c4c9472f95a645ec553bfa5a in this term.
I20260709 13:14:30.194084 11101 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.200235 11101 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.201954 10925 leader_election.cc:304] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08; no voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:30.196221 11082 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.203078 11130 raft_consensus.cc:2749] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.205598 10884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.202735 10886 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.194635 11124 raft_consensus.cc:697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:30.221535 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.223762 10960 raft_consensus.cc:2393] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a55dd39c4c9472f95a645ec553bfa5a in current term 1: Already voted for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in this term.
W20260709 13:14:30.224434 11000 leader_election.cc:343] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.225433 11119 raft_consensus.cc:515] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.229151 10884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.229858 10884 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.233598 11076 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: No bootstrap required, opened a new log
I20260709 13:14:30.234071 11076 ts_tablet_manager.cc:1403] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent bootstrapping tablet: real 0.147s	user 0.014s	sys 0.009s
I20260709 13:14:30.236647 11076 raft_consensus.cc:359] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.237289 11076 raft_consensus.cc:385] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.237551 11076 raft_consensus.cc:740] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.238104 11075 tablet_bootstrap.cc:654] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:30.238161 11076 consensus_queue.cc:260] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.240726 11076 ts_tablet_manager.cc:1434] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:30.243928 11124 consensus_queue.cc:237] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.253315 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.253993 10886 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.255345 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.255698 10884 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.256940 11000 leader_election.cc:304] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.257797 11082 raft_consensus.cc:2804] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.258248 11082 raft_consensus.cc:697] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:30.259080 11082 consensus_queue.cc:237] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.259500 10883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:30.260878 10882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:30.261075 10886 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.261075 10926 leader_election.cc:304] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08; no voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:30.262115 11083 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:30.262553 10926 leader_election.cc:343] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.262442 10784 catalog_manager.cc:5697] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:30.269831 11119 leader_election.cc:290] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.280462 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.281066 10959 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.281217 11083 raft_consensus.cc:2749] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.271059 10885 raft_consensus.cc:2468] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.286412 11000 leader_election.cc:304] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:30.287781 11000 leader_election.cc:304] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:30.288781 11116 raft_consensus.cc:2804] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.289117 11119 raft_consensus.cc:2804] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.289233 11116 raft_consensus.cc:697] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:30.290092 11116 consensus_queue.cc:237] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.290663 10959 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.274617 11120 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.292555 11117 leader_election.cc:290] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.293584 10960 raft_consensus.cc:2393] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a55dd39c4c9472f95a645ec553bfa5a in current term 1: Already voted for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in this term.
I20260709 13:14:30.294824 11120 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.297020 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:30.298043 11034 raft_consensus.cc:2393] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in current term 1: Already voted for candidate 7a55dd39c4c9472f95a645ec553bfa5a in this term.
I20260709 13:14:30.298056 11001 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.300022 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.300758 10960 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.304898 11101 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.311173 11117 raft_consensus.cc:2749] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.311278 10960 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.317044 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:30.316668 10780 catalog_manager.cc:5697] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:30.317838 11034 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.324515 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.328807 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:30.329816 11119 raft_consensus.cc:697] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:30.315049 11081 consensus_queue.cc:1048] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:14:30.330622 11119 consensus_queue.cc:237] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.331189 11095 consensus_queue.cc:1048] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:30.333009 11075 tablet_bootstrap.cc:492] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:30.333480 11075 ts_tablet_manager.cc:1403] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.146s	user 0.009s	sys 0.003s
I20260709 13:14:30.335691 11075 raft_consensus.cc:359] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.336367 11075 raft_consensus.cc:385] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.336645 11075 raft_consensus.cc:740] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
W20260709 13:14:30.336661 10926 leader_election.cc:343] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:30.337922 10926 leader_election.cc:343] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.338711 10783 catalog_manager.cc:5697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.338979 11075 consensus_queue.cc:260] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.340659 11101 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.341137 11075 ts_tablet_manager.cc:1434] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:30.341987 11075 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:30.344385 11034 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 1.
I20260709 13:14:30.345470 10925 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.346697 11101 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.347193 11101 raft_consensus.cc:697] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:30.348110 11075 tablet_bootstrap.cc:654] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:30.347952 11101 consensus_queue.cc:237] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.345505 10926 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08; no voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:30.352229 11120 raft_consensus.cc:2749] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.369836 10784 catalog_manager.cc:5697] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.371047 10780 catalog_manager.cc:5697] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.372967 10780 catalog_manager.cc:5697] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.375298 10780 catalog_manager.cc:5697] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.388769 11119 raft_consensus.cc:493] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.389300 11119 raft_consensus.cc:515] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.390472 11075 tablet_bootstrap.cc:492] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:30.390908 11075 ts_tablet_manager.cc:1403] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.023s
I20260709 13:14:30.391402 11119 leader_election.cc:290] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.392333 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
W20260709 13:14:30.393770 11000 leader_election.cc:343] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.393914 11075 raft_consensus.cc:359] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.394613 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.394834 11075 raft_consensus.cc:385] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.395226 11075 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.395326 10959 raft_consensus.cc:2468] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 0.
I20260709 13:14:30.396016 11075 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:30.398008 11001 leader_election.cc:304] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.398675 11075 ts_tablet_manager.cc:1434] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:14:30.399138 11119 raft_consensus.cc:2804] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:30.399529 11119 raft_consensus.cc:493] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.399900 11119 raft_consensus.cc:3060] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.399921 11075 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: Bootstrap starting.
I20260709 13:14:30.412128 11119 raft_consensus.cc:515] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.417280 11075 tablet_bootstrap.cc:654] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:30.418308 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
W20260709 13:14:30.419584 11000 leader_election.cc:343] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:30.420053 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:30.420990 10960 raft_consensus.cc:3060] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.422454 11075 tablet_bootstrap.cc:492] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: No bootstrap required, opened a new log
I20260709 13:14:30.422894 11075 ts_tablet_manager.cc:1403] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:14:30.423146 11119 leader_election.cc:290] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:30.425340 11075 raft_consensus.cc:359] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.425992 11075 raft_consensus.cc:385] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:30.426250 11075 raft_consensus.cc:740] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Initialized, Role: FOLLOWER
I20260709 13:14:30.426916 11075 consensus_queue.cc:260] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.427462 10960 raft_consensus.cc:2468] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 1.
I20260709 13:14:30.428582 11001 leader_election.cc:304] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:30.428956 11075 ts_tablet_manager.cc:1434] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:30.430029 11119 raft_consensus.cc:2804] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:30.430451 11119 raft_consensus.cc:697] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:30.431550 11119 consensus_queue.cc:237] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.440070 10784 catalog_manager.cc:5697] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 1 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.464358 11144 raft_consensus.cc:493] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.464803 11144 raft_consensus.cc:515] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.466424 11144 leader_election.cc:290] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.467227 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.467444 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f10cbaa44a3e4e0c99aa510e14fab02a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.467831 10960 raft_consensus.cc:2393] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fedf8972c484b14a503b9d9b498a317 in current term 1: Already voted for candidate 7a55dd39c4c9472f95a645ec553bfa5a in this term.
I20260709 13:14:30.468750 10852 leader_election.cc:304] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:30.469393 11144 raft_consensus.cc:3060] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.473914 11144 raft_consensus.cc:2749] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.536257 11101 consensus_queue.cc:1048] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.547703 11129 consensus_queue.cc:1048] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:30.667316 11144 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.667809 11144 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:30.669847 11144 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:30.670619 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" is_pre_election: true
I20260709 13:14:30.671017 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:30.671407 10959 raft_consensus.cc:2393] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fedf8972c484b14a503b9d9b498a317 in current term 1: Already voted for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in this term.
I20260709 13:14:30.672017 11035 raft_consensus.cc:2393] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fedf8972c484b14a503b9d9b498a317 in current term 1: Already voted for candidate 7a55dd39c4c9472f95a645ec553bfa5a in this term.
I20260709 13:14:30.672052 11144 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:30.672192 11119 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.673070 10850 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:30.673908 11144 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.681121 11144 raft_consensus.cc:2749] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:30.689769 11082 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.691601 11081 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.740048 11082 consensus_queue.cc:1048] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.740069 11129 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.752336 10886 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:30.752764 11135 consensus_queue.cc:1048] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.762341 11120 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:30.778703 11124 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.790514 11116 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.794963 11116 consensus_queue.cc:1048] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.807199 11119 consensus_queue.cc:1048] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:30.819329 10832 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:30.903537 11119 consensus_queue.cc:1048] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:30.913493 11082 consensus_queue.cc:1048] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:31.822587 10832 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:31.961730 11083 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.962240 11083 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:31.964159 11083 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:31.965137 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317" is_pre_election: true
I20260709 13:14:31.965238 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" is_pre_election: true
I20260709 13:14:31.965731 10886 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 1.
I20260709 13:14:31.965758 11034 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 1.
I20260709 13:14:31.966887 10926 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:31.967617 11167 raft_consensus.cc:2804] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:31.968012 11167 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.968290 11167 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:31.973085 11167 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:31.974756 11167 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:31.975523 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:31.975767 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:31.976029 10886 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:31.976272 11034 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:31.980566 10886 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:31.980648 11034 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:31.981699 10926 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:31.982353 11167 raft_consensus.cc:2804] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:31.982859 11167 raft_consensus.cc:697] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:31.983558 11167 consensus_queue.cc:237] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:31.990351 10784 catalog_manager.cc:5697] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 0 to 2, leader changed from <none> to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.370657 11034 raft_consensus.cc:1275] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:32.372059 11167 consensus_queue.cc:1048] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:32.388115 10886 raft_consensus.cc:1275] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:32.389775 11177 consensus_queue.cc:1048] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:32.656937 10959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:14:32.657665 10959 raft_consensus.cc:606] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:32.670589 10959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:14:32.671283 10959 raft_consensus.cc:606] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:32.683897 11034 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47309bbe949949e7823a02e151e39441"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:32.684360 11034 raft_consensus.cc:606] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:32.689512 11035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:32.690172 11035 raft_consensus.cc:606] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Received request to transfer leadership to TS 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:32.694962 11035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:32.695418 11035 raft_consensus.cc:606] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Received request to transfer leadership to TS 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:32.702651 11035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:32.703243 11035 raft_consensus.cc:606] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Received request to transfer leadership to TS 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:32.708127 11124 raft_consensus.cc:993] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:32.708575 11175 raft_consensus.cc:1081] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Signalling peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:32.710197 10959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:53712
I20260709 13:14:32.710721 10959 raft_consensus.cc:493] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.711055 10959 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.715801 10959 raft_consensus.cc:515] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.717579 10959 leader_election.cc:290] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:32.718389 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:32.718766 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "913b6d1c044f46c28c06b34cb355fd48" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:32.718930 10886 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.719333 11035 raft_consensus.cc:3055] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:32.719597 11035 raft_consensus.cc:740] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:32.720324 11035 consensus_queue.cc:260] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.721246 11035 raft_consensus.cc:3060] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.723825 10886 raft_consensus.cc:2468] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:32.724857 10926 leader_election.cc:304] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:32.725514 11083 raft_consensus.cc:2804] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:32.725909 11035 raft_consensus.cc:2468] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:32.725955 11083 raft_consensus.cc:697] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:32.726976 11083 consensus_queue.cc:237] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.734341 10784 catalog_manager.cc:5697] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 1 to 2, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.847934 11124 raft_consensus.cc:993] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:32.848434 11175 raft_consensus.cc:1081] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Signalling peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:32.849757 10959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:53712
I20260709 13:14:32.850241 10959 raft_consensus.cc:493] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.850555 10959 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.855754 10959 raft_consensus.cc:515] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.857108 11177 raft_consensus.cc:993] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.857576 11083 raft_consensus.cc:1081] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.857777 10959 leader_election.cc:290] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:32.858471 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:32.858732 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:32.859126 10886 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.859378 11035 raft_consensus.cc:3055] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:32.859565 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37278
I20260709 13:14:32.859759 11035 raft_consensus.cc:740] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:32.860134 10885 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.860495 10885 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.860632 11035 consensus_queue.cc:260] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.861848 11035 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.864194 10886 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:32.865202 10926 leader_election.cc:304] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:32.865907 11083 raft_consensus.cc:2804] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:32.866376 11083 raft_consensus.cc:697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:32.866935 10885 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:32.867694 11083 consensus_queue.cc:237] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.868239 11035 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:32.869086 10885 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Requested vote from peers 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:32.869841 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:32.870225 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:32.870381 11035 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.870721 10959 raft_consensus.cc:3055] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:32.871021 10959 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:32.871766 10959 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:32.872762 10959 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.877657 11035 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:32.877813 10959 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:32.878878 10850 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:32.879681 11166 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:32.881223 11166 raft_consensus.cc:697] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:32.882141 11166 consensus_queue.cc:237] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:32.882323 10784 catalog_manager.cc:5697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 1 to 2, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.889441 10780 catalog_manager.cc:5697] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 1 to 2, leader changed from 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.893954 11177 raft_consensus.cc:993] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.894441 11190 raft_consensus.cc:1081] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.897032 11124 raft_consensus.cc:993] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.897500 11124 raft_consensus.cc:1081] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:32.898139 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37278
I20260709 13:14:32.899430 10886 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47309bbe949949e7823a02e151e39441"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:14:32.898711 10885 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.899958 10886 raft_consensus.cc:493] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.900343 10885 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:32.900385 10886 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.905887 10885 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.905973 10886 raft_consensus.cc:515] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.907829 10885 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:32.908437 10886 leader_election.cc:290] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:32.908672 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:32.909160 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:32.909276 10959 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.909436 11035 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47309bbe949949e7823a02e151e39441" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:32.910223 11035 raft_consensus.cc:3055] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:32.910519 11035 raft_consensus.cc:740] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:32.911365 11035 consensus_queue.cc:260] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.909763 10960 raft_consensus.cc:3055] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:32.912663 10960 raft_consensus.cc:740] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:32.913682 10960 consensus_queue.cc:260] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.915457 11035 raft_consensus.cc:3060] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:32.915812 10959 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:32.916996 10852 leader_election.cc:304] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:32.917734 11166 raft_consensus.cc:2804] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:32.918169 11166 raft_consensus.cc:697] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:32.918771 11166 consensus_queue.cc:237] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.920280 10960 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:32.925320 10960 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:32.926458 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ab2b71d68a940deba12d61b570cec68" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:32.926501 10852 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:32.927157 11034 raft_consensus.cc:3060] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:32.927695 11166 raft_consensus.cc:2804] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:32.928427 11166 raft_consensus.cc:697] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:32.929276 11166 consensus_queue.cc:237] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:32.930905 11035 raft_consensus.cc:2468] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:32.930596 10780 catalog_manager.cc:5697] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 1 to 2, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.933527 11034 raft_consensus.cc:2468] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:32.942592 10780 catalog_manager.cc:5697] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 2 to 3, leader changed from 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:33.125445 11175 raft_consensus.cc:993] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:33.125883 11124 raft_consensus.cc:1081] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Signalling peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:33.127213 10960 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:53712
I20260709 13:14:33.127610 10960 raft_consensus.cc:493] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:33.127924 10960 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:33.133490 10960 raft_consensus.cc:515] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:33.135102 10960 leader_election.cc:290] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:33.135792 10886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:33.136016 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:33.136322 10886 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:33.136613 11034 raft_consensus.cc:3055] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:33.136912 11034 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:33.137516 11034 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:33.138414 11034 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:33.140897 10886 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:33.141896 10926 leader_election.cc:304] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:33.142540 11177 raft_consensus.cc:2804] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:33.142831 11034 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 2.
I20260709 13:14:33.142899 11177 raft_consensus.cc:697] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:33.143885 11177 consensus_queue.cc:237] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:33.150645 10780 catalog_manager.cc:5697] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 1 to 2, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:33.192884 11034 raft_consensus.cc:1275] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.194089 11177 consensus_queue.cc:1048] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.203341 10886 raft_consensus.cc:1275] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.204914 11190 consensus_queue.cc:1048] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.280042 10886 raft_consensus.cc:1275] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.281265 11190 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.289326 11034 raft_consensus.cc:1275] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.291430 11177 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.310503 11034 raft_consensus.cc:1275] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:33.311769 11166 consensus_queue.cc:1048] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.319391 10960 raft_consensus.cc:1275] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:33.320967 11188 consensus_queue.cc:1048] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.437435 10960 raft_consensus.cc:1275] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.438680 11188 consensus_queue.cc:1048] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.443699 11034 raft_consensus.cc:1275] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.445309 11200 consensus_queue.cc:1048] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.486652 11034 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.487910 11166 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.496556 10960 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.498219 11166 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.555369 10886 raft_consensus.cc:1275] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.556619 11177 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:33.564857 11034 raft_consensus.cc:1275] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:33.566356 11190 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:35.736528 10252 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:14:35.737193 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:35.758688 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:35.759362 10885 raft_consensus.cc:606] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Received request to transfer leadership to TS 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:35.761859 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:35.762284 10885 raft_consensus.cc:606] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Received request to transfer leadership to TS 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:35.763161 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:35.763540 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:35.868403 11210 raft_consensus.cc:993] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:35.869383 11188 raft_consensus.cc:1081] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Signalling peer 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:35.871183 11035 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:14:35.871731 11035 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:35.872110 11035 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:35.880081 11035 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:35.881841 11035 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 3 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:35.882596 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:35.882917 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:35.883148 10885 raft_consensus.cc:3055] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:35.883486 10885 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:35.883496 10960 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:35.884258 10885 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:35.885386 10885 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:35.889123 10960 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 3.
I20260709 13:14:35.890165 11001 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:35.890347 10885 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 3.
I20260709 13:14:35.890975 11175 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:35.892072 11175 raft_consensus.cc:697] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:35.892791 11175 consensus_queue.cc:237] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:35.899683 10780 catalog_manager.cc:5697] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 2 to 3, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 3 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:36.278820 10960 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:36.280434 11175 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.292057 10885 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:36.294289 11222 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.324533 11210 raft_consensus.cc:993] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:36.325335 11188 raft_consensus.cc:1081] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Signalling peer 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:36.327046 11035 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:14:36.327692 11035 raft_consensus.cc:493] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:36.328073 11035 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.333528 11035 raft_consensus.cc:515] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:36.335270 11035 leader_election.cc:290] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:36.336184 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:36.336386 10960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:36.336942 10960 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.336925 10885 raft_consensus.cc:3055] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:36.337481 10885 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:36.338457 10885 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:36.339880 10885 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:36.344836 10960 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 2.
I20260709 13:14:36.346239 11001 leader_election.cc:304] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:36.346925 10885 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 2.
I20260709 13:14:36.347126 11222 raft_consensus.cc:2804] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:36.347579 11222 raft_consensus.cc:697] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:36.348513 11222 consensus_queue.cc:237] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:36.355363 10780 catalog_manager.cc:5697] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 1 to 2, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 2 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.764488 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:36.767536 10885 raft_consensus.cc:1275] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.768972 11222 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.779083 10960 raft_consensus.cc:1275] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:36.780689 11175 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:36.780522 11035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:36.781883 11035 raft_consensus.cc:606] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:36.794140 11035 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:46198
I20260709 13:14:36.795290 11035 raft_consensus.cc:606] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:36.807121 10960 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:14:36.807657 10960 raft_consensus.cc:606] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:36.812644 10960 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:14:36.813218 10960 raft_consensus.cc:606] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:36.818007 10959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
mode: GRACEFUL
new_leader_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:14:36.818555 10959 raft_consensus.cc:606] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Received request to transfer leadership to TS 6fedf8972c484b14a503b9d9b498a317
I20260709 13:14:36.824060 11222 raft_consensus.cc:993] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:36.824546 11175 raft_consensus.cc:1081] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:36.826417 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:14:36.826879 10885 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:36.827235 10885 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:36.831676 10885 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:36.833818 10885 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 4 election: Requested vote from peers 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:36.834205 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:36.834398 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:36.834810 11034 raft_consensus.cc:3055] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:36.834873 10959 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:36.835278 11034 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:36.836123 11034 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:36.837240 11034 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:36.841969 10959 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 4.
I20260709 13:14:36.843168 10852 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:36.843580 11034 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 4.
I20260709 13:14:36.843833 11188 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:36.844980 11188 raft_consensus.cc:697] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:36.845701 11188 consensus_queue.cc:237] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:36.850899 10780 catalog_manager.cc:5697] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 3 to 4, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.071056 11230 raft_consensus.cc:993] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.071480 11230 raft_consensus.cc:1081] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.072906 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37278
I20260709 13:14:37.073383 10885 raft_consensus.cc:493] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.073724 10885 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:37.078348 10885 raft_consensus.cc:515] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.080137 10885 leader_election.cc:290] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:37.080919 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:37.081099 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5be0ff321734d5ba0eb8782c3d0fb0c" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:37.081494 10959 raft_consensus.cc:3055] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:37.081714 11034 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:37.081840 10959 raft_consensus.cc:740] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:37.082674 10959 consensus_queue.cc:260] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.083721 10959 raft_consensus.cc:3060] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:37.086186 11034 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:37.087365 10850 leader_election.cc:304] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:37.088208 11188 raft_consensus.cc:2804] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:37.088310 10959 raft_consensus.cc:2468] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 2.
I20260709 13:14:37.088728 11188 raft_consensus.cc:697] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:37.089584 11188 consensus_queue.cc:237] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.095911 10780 catalog_manager.cc:5697] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 1 to 2, leader changed from 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.203500 11219 raft_consensus.cc:993] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.203898 11230 raft_consensus.cc:1081] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.205264 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37278
I20260709 13:14:37.205677 10885 raft_consensus.cc:493] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.205943 10885 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.210649 10885 raft_consensus.cc:515] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.212142 10885 leader_election.cc:290] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:37.212988 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:37.213135 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:37.213624 10959 raft_consensus.cc:3055] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.213624 11034 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.214200 10959 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:37.215057 10959 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.216255 10959 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.220230 11034 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.221299 10850 leader_election.cc:304] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:37.222028 11188 raft_consensus.cc:2804] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.222503 11188 raft_consensus.cc:697] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:37.222486 10959 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.223634 11188 consensus_queue.cc:237] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.230114 10780 catalog_manager.cc:5697] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 2 to 3, leader changed from 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.287375 11222 raft_consensus.cc:993] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.287803 11175 raft_consensus.cc:1081] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.289266 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:14:37.289762 10885 raft_consensus.cc:493] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.290053 10885 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.294869 10885 raft_consensus.cc:515] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.296665 10885 leader_election.cc:290] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:37.297416 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:37.297699 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:37.298089 10959 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.298238 11034 raft_consensus.cc:3055] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.298578 11034 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:37.299381 11034 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.300200 11034 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.302569 10959 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.303541 10852 leader_election.cc:304] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:37.304294 11188 raft_consensus.cc:2804] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.304425 11034 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.304738 11188 raft_consensus.cc:697] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:37.305569 11188 consensus_queue.cc:237] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.311854 10780 catalog_manager.cc:5697] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 2 to 3, leader changed from 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.328963 11034 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:37.330080 11188 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:37.340905 10959 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:37.342389 11188 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:37.412118 11219 raft_consensus.cc:993] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: : Instructing follower 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.412563 11219 raft_consensus.cc:1081] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Signalling peer 6fedf8972c484b14a503b9d9b498a317 to start an election
I20260709 13:14:37.414109 10885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
 from {username='slave'} at 127.0.0.1:37278
I20260709 13:14:37.414615 10885 raft_consensus.cc:493] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.414925 10885 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.419319 10885 raft_consensus.cc:515] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.420611 10885 leader_election.cc:290] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Requested vote from peers 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:37.421489 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:37.421758 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474f8e3b3e29485a8bc3ec260fa7c69d" candidate_uuid: "6fedf8972c484b14a503b9d9b498a317" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:37.421960 10959 raft_consensus.cc:3055] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:37.422262 10959 raft_consensus.cc:740] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:37.422282 11034 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.423055 10959 consensus_queue.cc:260] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.424122 10959 raft_consensus.cc:3060] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:37.426453 11034 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.427419 10850 leader_election.cc:304] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:37.428031 11188 raft_consensus.cc:2804] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:37.428457 11188 raft_consensus.cc:697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Leader. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:37.428773 10959 raft_consensus.cc:2468] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fedf8972c484b14a503b9d9b498a317 in term 3.
I20260709 13:14:37.429272 11188 consensus_queue.cc:237] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:37.436806 10783 catalog_manager.cc:5697] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 reported cstate change: term changed from 2 to 3, leader changed from 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2) to 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "6fedf8972c484b14a503b9d9b498a317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.474805 10959 raft_consensus.cc:1275] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:37.476032 11188 consensus_queue.cc:1048] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:37.484108 11034 raft_consensus.cc:1275] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:37.485719 11229 consensus_queue.cc:1048] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:37.630141 11034 raft_consensus.cc:1275] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.631433 11229 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.639732 10959 raft_consensus.cc:1275] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.641647 11234 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:37.745980 11034 raft_consensus.cc:1275] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.747148 11234 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.753995 10959 raft_consensus.cc:1275] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.755398 11234 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.829170 11034 raft_consensus.cc:1275] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.830283 11234 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:37.841154 10959 raft_consensus.cc:1275] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Refusing update from remote peer 6fedf8972c484b14a503b9d9b498a317: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:37.842526 11234 consensus_queue.cc:1048] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:14:39.561129 10982 debug-util.cc:398] Leaking SignalData structure 0x7b0800192500 after lost signal to thread 10255
W20260709 13:14:39.562194 10982 debug-util.cc:398] Leaking SignalData structure 0x7b080019f1e0 after lost signal to thread 10818
W20260709 13:14:39.563153 10982 debug-util.cc:398] Leaking SignalData structure 0x7b0800098640 after lost signal to thread 10910
W20260709 13:14:39.563812 10982 debug-util.cc:398] Leaking SignalData structure 0x7b080017e0e0 after lost signal to thread 10985
W20260709 13:14:39.564460 10982 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbf20 after lost signal to thread 11059
I20260709 13:14:39.935075 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:39.943408 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:39.944002 10885 raft_consensus.cc:606] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Received request to transfer leadership to TS 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:39.946755 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:39.947328 10885 raft_consensus.cc:606] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Received request to transfer leadership to TS 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:39.949674 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:39.950091 10885 raft_consensus.cc:606] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Received request to transfer leadership to TS 7a55dd39c4c9472f95a645ec553bfa5a
I20260709 13:14:39.952333 10885 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
mode: GRACEFUL
new_leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:37290
I20260709 13:14:39.952806 10885 raft_consensus.cc:606] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 LEADER]: Received request to transfer leadership to TS 5611bbb1b2ca4dbd8a8f6b7109a16f08
I20260709 13:14:39.953778 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:14:39.954231 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:39.980230 11245 raft_consensus.cc:993] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:39.980814 11247 raft_consensus.cc:1081] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 LEADER]: Signalling peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:39.982184 10959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:53708
I20260709 13:14:39.982694 10959 raft_consensus.cc:493] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:39.983035 10959 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:39.987857 10959 raft_consensus.cc:515] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:39.989287 10959 leader_election.cc:290] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 5 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:39.990306 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:39.990298 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6febcca359294eeba020eb05b75c5eee" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:39.990953 11034 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:39.991155 10885 raft_consensus.cc:3055] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:39.991616 10885 raft_consensus.cc:740] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:39.992434 10885 consensus_queue.cc:260] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:39.993788 10885 raft_consensus.cc:3060] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:39.995419 11034 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 5.
I20260709 13:14:39.996521 10925 leader_election.cc:304] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:39.997226 11219 raft_consensus.cc:2804] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:40.001603 10885 raft_consensus.cc:2468] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 5.
I20260709 13:14:40.003042 11219 raft_consensus.cc:697] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 5 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:40.003849 11219 consensus_queue.cc:237] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } }
I20260709 13:14:40.014866 10783 catalog_manager.cc:5697] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 4 to 5, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 5 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.173527 11245 raft_consensus.cc:993] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:40.174108 11247 raft_consensus.cc:1081] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Signalling peer 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:40.176059 11034 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:14:40.176723 11034 raft_consensus.cc:493] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:40.177237 11034 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:40.182302 11034 raft_consensus.cc:515] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.183871 11034 leader_election.cc:290] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 2 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:40.184715 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:40.184903 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a682a9d136e4a1d8549766f16b0dcf7" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:40.185281 10885 raft_consensus.cc:3055] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:40.185333 10959 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:40.185676 10885 raft_consensus.cc:740] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:40.186475 10885 consensus_queue.cc:260] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.187633 10885 raft_consensus.cc:3060] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:40.191435 10959 raft_consensus.cc:2468] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 2.
I20260709 13:14:40.192624 11001 leader_election.cc:304] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:40.193406 10885 raft_consensus.cc:2468] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 2.
I20260709 13:14:40.193569 11222 raft_consensus.cc:2804] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:40.195024 11222 raft_consensus.cc:697] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:40.195827 11222 consensus_queue.cc:237] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.205205 10783 catalog_manager.cc:5697] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 1 to 2, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 2 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.384814 11034 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 5 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:40.386152 11219 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:40.387158 11245 raft_consensus.cc:993] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:40.387599 11247 raft_consensus.cc:1081] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Signalling peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 to start an election
I20260709 13:14:40.389083 10959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce"
dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
 from {username='slave'} at 127.0.0.1:53708
I20260709 13:14:40.389607 10959 raft_consensus.cc:493] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:40.390533 10959 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.398726 10959 raft_consensus.cc:515] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.401743 10959 leader_election.cc:290] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 4 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3:46831)
I20260709 13:14:40.403337 10885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:40.403921 10885 raft_consensus.cc:3055] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:40.404201 10885 raft_consensus.cc:740] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:40.404924 10885 consensus_queue.cc:260] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.406191 10885 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.412178 11245 raft_consensus.cc:993] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: : Instructing follower 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:40.412707 11245 raft_consensus.cc:1081] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Signalling peer 7a55dd39c4c9472f95a645ec553bfa5a to start an election
I20260709 13:14:40.414096 11034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307db797d62949eba0c77576ca9ae5ce" candidate_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
I20260709 13:14:40.414760 11035 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0908a580b50644668401a9ecb02d481a"
dest_uuid: "7a55dd39c4c9472f95a645ec553bfa5a"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:14:40.415581 11034 raft_consensus.cc:3060] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.415664 11035 raft_consensus.cc:493] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:40.416172 11035 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.420073 10886 raft_consensus.cc:1275] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 5 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:40.427088 11219 consensus_queue.cc:1048] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:40.444974 10885 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 4.
I20260709 13:14:40.437032 11035 raft_consensus.cc:515] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.448559 10926 leader_election.cc:304] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317; no voters: 
I20260709 13:14:40.450556 11219 raft_consensus.cc:2804] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:40.451493 11219 raft_consensus.cc:697] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 LEADER]: Becoming Leader. State: Replica: 5611bbb1b2ca4dbd8a8f6b7109a16f08, State: Running, Role: LEADER
I20260709 13:14:40.451524 10882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6fedf8972c484b14a503b9d9b498a317"
I20260709 13:14:40.452684 10882 raft_consensus.cc:3055] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:40.453146 10882 raft_consensus.cc:740] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6fedf8972c484b14a503b9d9b498a317, State: Running, Role: LEADER
I20260709 13:14:40.454213 10959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0908a580b50644668401a9ecb02d481a" candidate_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08"
I20260709 13:14:40.454917 10959 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.454387 10882 consensus_queue.cc:260] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.463393 10882 raft_consensus.cc:3060] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:40.474113 10882 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 4.
I20260709 13:14:40.453142 11035 leader_election.cc:290] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 election: Requested vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:40.455884 11219 consensus_queue.cc:237] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.482120 10959 raft_consensus.cc:2468] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55dd39c4c9472f95a645ec553bfa5a in term 4.
I20260709 13:14:40.487920 11000 leader_election.cc:304] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fedf8972c484b14a503b9d9b498a317, 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 
I20260709 13:14:40.489073 11222 raft_consensus.cc:2804] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:40.489552 11034 raft_consensus.cc:2468] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5611bbb1b2ca4dbd8a8f6b7109a16f08 in term 4.
I20260709 13:14:40.489637 11222 raft_consensus.cc:697] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 LEADER]: Becoming Leader. State: Replica: 7a55dd39c4c9472f95a645ec553bfa5a, State: Running, Role: LEADER
I20260709 13:14:40.492926 11222 consensus_queue.cc:237] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:40.505362 10783 catalog_manager.cc:5697] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 reported cstate change: term changed from 3 to 4, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2). New cstate: current_term: 4 leader_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:40.515197 10780 catalog_manager.cc:5697] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a reported cstate change: term changed from 3 to 4, leader changed from 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1) to 7a55dd39c4c9472f95a645ec553bfa5a (127.10.3.3). New cstate: current_term: 4 leader_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.627394 10959 raft_consensus.cc:1275] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:40.628659 11222 consensus_queue.cc:1048] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:40.638456 10886 raft_consensus.cc:1275] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:40.641448 11254 consensus_queue.cc:1048] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:40.955101 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:40.961473 10252 tablet_server.cc:179] TabletServer@127.10.3.1:0 shutting down...
W20260709 13:14:40.981595 10926 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:14:40.995087 10926 consensus_peers.cc:597] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 -> Peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Couldn't send request to peer 6fedf8972c484b14a503b9d9b498a317. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:40.995764 11034 raft_consensus.cc:1275] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Refusing update from remote peer 5611bbb1b2ca4dbd8a8f6b7109a16f08: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:40.996922 11259 consensus_queue.cc:1048] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:41.000756 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:41.002213 10252 tablet_replica.cc:333] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.003170 10252 raft_consensus.cc:2243] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.003809 10252 raft_consensus.cc:2272] T f10cbaa44a3e4e0c99aa510e14fab02a P 6fedf8972c484b14a503b9d9b498a317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.007787 10252 tablet_replica.cc:333] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.008610 10252 raft_consensus.cc:2243] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.009145 10252 raft_consensus.cc:2272] T 307db797d62949eba0c77576ca9ae5ce P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.011734 10252 tablet_replica.cc:333] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.012485 10252 raft_consensus.cc:2243] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.013521 10252 raft_consensus.cc:2272] T 7ab2b71d68a940deba12d61b570cec68 P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.016114 10252 tablet_replica.cc:333] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.016884 10252 raft_consensus.cc:2243] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.017848 10252 raft_consensus.cc:2272] T 474f8e3b3e29485a8bc3ec260fa7c69d P 6fedf8972c484b14a503b9d9b498a317 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.020327 10252 tablet_replica.cc:333] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.021178 10252 raft_consensus.cc:2243] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.021752 10252 raft_consensus.cc:2272] T 6a682a9d136e4a1d8549766f16b0dcf7 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.024945 10252 tablet_replica.cc:333] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.025700 10252 raft_consensus.cc:2243] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.026651 10252 raft_consensus.cc:2272] T c5be0ff321734d5ba0eb8782c3d0fb0c P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.029105 10252 tablet_replica.cc:333] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.029839 10252 raft_consensus.cc:2243] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.030340 10252 raft_consensus.cc:2272] T 6febcca359294eeba020eb05b75c5eee P 6fedf8972c484b14a503b9d9b498a317 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.033087 10252 tablet_replica.cc:333] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.033881 10252 raft_consensus.cc:2243] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.034379 10252 raft_consensus.cc:2272] T 913b6d1c044f46c28c06b34cb355fd48 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.036849 10252 tablet_replica.cc:333] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.037509 10252 raft_consensus.cc:2243] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.038436 10252 raft_consensus.cc:2272] T 47309bbe949949e7823a02e151e39441 P 6fedf8972c484b14a503b9d9b498a317 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.040825 10252 tablet_replica.cc:333] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317: stopping tablet replica
I20260709 13:14:41.041468 10252 raft_consensus.cc:2243] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.041925 10252 raft_consensus.cc:2272] T 0908a580b50644668401a9ecb02d481a P 6fedf8972c484b14a503b9d9b498a317 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.066749 10959 raft_consensus.cc:1275] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Refusing update from remote peer 7a55dd39c4c9472f95a645ec553bfa5a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:41.068233 11254 consensus_queue.cc:1048] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:41.068764 10252 tablet_server.cc:196] TabletServer@127.10.3.1:0 shutdown complete.
I20260709 13:14:41.106626 10252 tablet_server.cc:179] TabletServer@127.10.3.2:0 shutting down...
W20260709 13:14:41.150624 10926 consensus_peers.cc:597] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 -> Peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Couldn't send request to peer 6fedf8972c484b14a503b9d9b498a317. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:41007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.156391 11000 consensus_peers.cc:597] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Couldn't send request to peer 6fedf8972c484b14a503b9d9b498a317. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:41007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.162245 11001 consensus_peers.cc:597] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Couldn't send request to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.163192 11000 consensus_peers.cc:597] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Couldn't send request to peer 6fedf8972c484b14a503b9d9b498a317. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:41007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.163774 11000 consensus_peers.cc:597] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Couldn't send request to peer 6fedf8972c484b14a503b9d9b498a317. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:41007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.164040 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:41.164815 10252 tablet_replica.cc:333] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.165727 10252 raft_consensus.cc:2243] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.166299 10252 raft_consensus.cc:2272] T 6a682a9d136e4a1d8549766f16b0dcf7 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.169234 10252 tablet_replica.cc:333] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.169955 10252 raft_consensus.cc:2243] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.170449 10252 raft_consensus.cc:2272] T c5be0ff321734d5ba0eb8782c3d0fb0c P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.172775 10252 tablet_replica.cc:333] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.173468 10252 raft_consensus.cc:2243] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.173975 10252 raft_consensus.cc:2272] T 7ab2b71d68a940deba12d61b570cec68 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.176297 10252 tablet_replica.cc:333] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.177014 10252 raft_consensus.cc:2243] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.177520 10252 raft_consensus.cc:2272] T 474f8e3b3e29485a8bc3ec260fa7c69d P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.179828 10252 tablet_replica.cc:333] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.180559 10252 raft_consensus.cc:2243] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.181576 10252 raft_consensus.cc:2272] T 307db797d62949eba0c77576ca9ae5ce P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.184219 10252 tablet_replica.cc:333] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.184909 10252 raft_consensus.cc:2243] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.185446 10252 raft_consensus.cc:2272] T f10cbaa44a3e4e0c99aa510e14fab02a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.187849 10252 tablet_replica.cc:333] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.188570 10252 raft_consensus.cc:2243] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.189486 10252 raft_consensus.cc:2272] T 6febcca359294eeba020eb05b75c5eee P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.191946 10252 tablet_replica.cc:333] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.192658 10252 raft_consensus.cc:2243] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.193636 10252 raft_consensus.cc:2272] T 913b6d1c044f46c28c06b34cb355fd48 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.195948 10252 tablet_replica.cc:333] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.196593 10252 raft_consensus.cc:2243] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.197072 10252 raft_consensus.cc:2272] T 47309bbe949949e7823a02e151e39441 P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.199525 10252 tablet_replica.cc:333] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08: stopping tablet replica
I20260709 13:14:41.200153 10252 raft_consensus.cc:2243] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.200686 10252 raft_consensus.cc:2272] T 0908a580b50644668401a9ecb02d481a P 5611bbb1b2ca4dbd8a8f6b7109a16f08 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:41.313812 11001 consensus_peers.cc:597] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Couldn't send request to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:38059: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:41.567776 11001 consensus_peers.cc:597] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a -> Peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Couldn't send request to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:38059: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:42.076092 10252 tablet_server.cc:196] TabletServer@127.10.3.2:0 shutdown complete.
I20260709 13:14:42.097297 10252 tablet_server.cc:179] TabletServer@127.10.3.3:0 shutting down...
I20260709 13:14:42.116295 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:42.117007 10252 tablet_replica.cc:333] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.117611 10252 raft_consensus.cc:2243] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:42.118494 10252 raft_consensus.cc:2272] T 0908a580b50644668401a9ecb02d481a P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.120193 10252 tablet_replica.cc:333] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.120683 10252 raft_consensus.cc:2243] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.121079 10252 raft_consensus.cc:2272] T 6febcca359294eeba020eb05b75c5eee P 7a55dd39c4c9472f95a645ec553bfa5a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.122515 10252 tablet_replica.cc:333] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.123075 10252 raft_consensus.cc:2243] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.123462 10252 raft_consensus.cc:2272] T 913b6d1c044f46c28c06b34cb355fd48 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.124805 11254 raft_consensus.cc:493] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6fedf8972c484b14a503b9d9b498a317)
I20260709 13:14:42.125100 10252 tablet_replica.cc:333] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.125257 11254 raft_consensus.cc:515] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fedf8972c484b14a503b9d9b498a317" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 41007 } } peers { permanent_uuid: "5611bbb1b2ca4dbd8a8f6b7109a16f08" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 38059 } } peers { permanent_uuid: "7a55dd39c4c9472f95a645ec553bfa5a" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46831 } }
I20260709 13:14:42.125630 10252 raft_consensus.cc:2243] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.126114 10252 raft_consensus.cc:2272] T 47309bbe949949e7823a02e151e39441 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.127151 11254 leader_election.cc:290] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007), 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059)
I20260709 13:14:42.128065 10252 tablet_replica.cc:333] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.128579 10252 raft_consensus.cc:2243] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.128975 10252 raft_consensus.cc:2272] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.131326 10252 tablet_replica.cc:333] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.132066 10252 raft_consensus.cc:2243] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:42.132164 11000 leader_election.cc:336] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6fedf8972c484b14a503b9d9b498a317 (127.10.3.1:41007): Network error: Client connection negotiation failed: client connection to 127.10.3.1:41007: connect: Connection refused (error 111)
I20260709 13:14:42.132562 10252 raft_consensus.cc:2272] T 474f8e3b3e29485a8bc3ec260fa7c69d P 7a55dd39c4c9472f95a645ec553bfa5a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:42.134223 11001 leader_election.cc:336] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5611bbb1b2ca4dbd8a8f6b7109a16f08 (127.10.3.2:38059): Network error: Client connection negotiation failed: client connection to 127.10.3.2:38059: connect: Connection refused (error 111)
I20260709 13:14:42.134618 10252 tablet_replica.cc:333] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.134565 11001 leader_election.cc:304] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a55dd39c4c9472f95a645ec553bfa5a; no voters: 5611bbb1b2ca4dbd8a8f6b7109a16f08, 6fedf8972c484b14a503b9d9b498a317
W20260709 13:14:42.135180 11001 raft_consensus.cc:2686] T 7ab2b71d68a940deba12d61b570cec68 P 7a55dd39c4c9472f95a645ec553bfa5a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:14:42.135274 10252 raft_consensus.cc:2243] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:42.136046 10252 raft_consensus.cc:2272] T 6a682a9d136e4a1d8549766f16b0dcf7 P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.137633 10252 tablet_replica.cc:333] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.138093 10252 raft_consensus.cc:2243] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.138468 10252 raft_consensus.cc:2272] T c5be0ff321734d5ba0eb8782c3d0fb0c P 7a55dd39c4c9472f95a645ec553bfa5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.140740 10252 tablet_replica.cc:333] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.141245 10252 raft_consensus.cc:2243] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:42.142069 10252 raft_consensus.cc:2272] T f10cbaa44a3e4e0c99aa510e14fab02a P 7a55dd39c4c9472f95a645ec553bfa5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.143692 10252 tablet_replica.cc:333] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a: stopping tablet replica
I20260709 13:14:42.144061 10252 raft_consensus.cc:2243] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.144388 10252 raft_consensus.cc:2272] T 307db797d62949eba0c77576ca9ae5ce P 7a55dd39c4c9472f95a645ec553bfa5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.174261 10252 tablet_server.cc:196] TabletServer@127.10.3.3:0 shutdown complete.
I20260709 13:14:42.192214 10252 master.cc:562] Master@127.10.3.62:32873 shutting down...
I20260709 13:14:42.211784 10252 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:42.212319 10252 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.212641 10252 tablet_replica.cc:333] T 00000000000000000000000000000000 P 564ba8ab7cbf4e1191d4a095bf9bc9a9: stopping tablet replica
I20260709 13:14:42.230906 10252 master.cc:584] Master@127.10.3.62:32873 shutdown complete.
5611bbb1b2ca4dbd8a8f6b7109a16f08  4
6fedf8972c484b14a503b9d9b498a317  5
7a55dd39c4c9472f95a645ec553bfa5a  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (13586 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:14:42.270076 10252 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.3.62:34815
I20260709 13:14:42.271288 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:42.277096 11276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.277148 11275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.278040 11278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.278424 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:42.279347 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:42.279551 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:42.279680 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602882279666 us; error 0 us; skew 500 ppm
I20260709 13:14:42.280135 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:42.282423 10252 webserver.cc:533] Webserver started at http://127.10.3.62:35563/ using document root <none> and password file <none>
I20260709 13:14:42.282838 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:42.283224 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:42.283473 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:42.284520 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/master-0-root/instance:
uuid: "d085878dd61a4a3984dff72a53c92b4d"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.288766 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:14:42.292003 11283 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.292692 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:42.292975 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/master-0-root
uuid: "d085878dd61a4a3984dff72a53c92b4d"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.293208 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:42.309732 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:42.310951 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:42.348171 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.62:34815
I20260709 13:14:42.348292 11334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.62:34815 every 8 connection(s)
I20260709 13:14:42.351923 11335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.361771 11335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Bootstrap starting.
I20260709 13:14:42.366279 11335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:42.370440 11335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: No bootstrap required, opened a new log
I20260709 13:14:42.372509 11335 raft_consensus.cc:359] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER }
I20260709 13:14:42.372933 11335 raft_consensus.cc:385] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:42.373154 11335 raft_consensus.cc:740] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d085878dd61a4a3984dff72a53c92b4d, State: Initialized, Role: FOLLOWER
I20260709 13:14:42.373793 11335 consensus_queue.cc:260] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER }
I20260709 13:14:42.374230 11335 raft_consensus.cc:399] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:42.374464 11335 raft_consensus.cc:493] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:42.374722 11335 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:42.378827 11335 raft_consensus.cc:515] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER }
I20260709 13:14:42.379364 11335 leader_election.cc:304] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d085878dd61a4a3984dff72a53c92b4d; no voters: 
I20260709 13:14:42.380529 11335 leader_election.cc:290] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:42.380846 11338 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:42.382157 11338 raft_consensus.cc:697] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [term 1 LEADER]: Becoming Leader. State: Replica: d085878dd61a4a3984dff72a53c92b4d, State: Running, Role: LEADER
I20260709 13:14:42.382833 11338 consensus_queue.cc:237] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER }
I20260709 13:14:42.383565 11335 sys_catalog.cc:565] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:42.387719 11340 sys_catalog.cc:455] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d085878dd61a4a3984dff72a53c92b4d. Latest consensus state: current_term: 1 leader_uuid: "d085878dd61a4a3984dff72a53c92b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER } }
I20260709 13:14:42.388001 11339 sys_catalog.cc:455] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d085878dd61a4a3984dff72a53c92b4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d085878dd61a4a3984dff72a53c92b4d" member_type: VOTER } }
I20260709 13:14:42.388423 11340 sys_catalog.cc:458] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:42.388703 11339 sys_catalog.cc:458] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:42.391671 11345 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:42.396469 11345 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:42.405400 11345 catalog_manager.cc:1383] Generated new cluster ID: 3ff2d7494a944ce09ab4d8df78dd2940
I20260709 13:14:42.405710 11345 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:42.406036 10252 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:42.430159 11345 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:42.431430 11345 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:42.444407 11345 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Generated new TSK 0
I20260709 13:14:42.445014 11345 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:42.472134 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:42.478240 11356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.479892 11357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.481408 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:42.481606 11359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.482549 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:42.482755 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:42.482877 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602882482864 us; error 0 us; skew 500 ppm
I20260709 13:14:42.483409 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:42.485747 10252 webserver.cc:533] Webserver started at http://127.10.3.1:36193/ using document root <none> and password file <none>
I20260709 13:14:42.486157 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:42.486308 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:42.486513 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:42.487576 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/instance:
uuid: "147201d22a694ca59f6e014f083a0892"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.491760 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:14:42.494830 11364 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.495644 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:42.495965 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root
uuid: "147201d22a694ca59f6e014f083a0892"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.496263 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:42.522552 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:42.523908 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:42.525408 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:42.527675 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:42.527911 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.528160 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:42.528349 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.568233 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.1:37459
I20260709 13:14:42.568337 11426 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.1:37459 every 8 connection(s)
I20260709 13:14:42.573079 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:42.580114 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.581965 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.583199 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.584654 10252 server_base.cc:1061] running on GCE node
I20260709 13:14:42.585505 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:42.585734 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:42.585920 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602882585903 us; error 0 us; skew 500 ppm
I20260709 13:14:42.586426 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:42.588691 10252 webserver.cc:533] Webserver started at http://127.10.3.2:33789/ using document root <none> and password file <none>
I20260709 13:14:42.589386 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:42.589614 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:42.589936 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:42.591506 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/instance:
uuid: "34fe1737ebb840fb9746980f649cc0a6"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.592245 11427 heartbeater.cc:344] Connected to a master server at 127.10.3.62:34815
I20260709 13:14:42.592608 11427 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:42.593309 11427 heartbeater.cc:507] Master 127.10.3.62:34815 requested a full tablet report, sending...
I20260709 13:14:42.595520 11300 ts_manager.cc:194] Registered new tserver with Master: 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:14:42.596889 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:14:42.597955 11300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34858
I20260709 13:14:42.601177 11439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.602051 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:14:42.602290 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root
uuid: "34fe1737ebb840fb9746980f649cc0a6"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.602525 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:42.614475 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:42.615654 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:42.616986 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:42.619148 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:42.619349 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.619601 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:42.619777 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.661279 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.2:36161
I20260709 13:14:42.661399 11501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.2:36161 every 8 connection(s)
I20260709 13:14:42.666779 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:42.675608 11505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:42.677424 11506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.678659 11502 heartbeater.cc:344] Connected to a master server at 127.10.3.62:34815
I20260709 13:14:42.679147 11502 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:42.679297 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:42.680014 11508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:42.680148 11502 heartbeater.cc:507] Master 127.10.3.62:34815 requested a full tablet report, sending...
I20260709 13:14:42.680924 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:42.681169 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:42.681344 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602882681327 us; error 0 us; skew 500 ppm
I20260709 13:14:42.682024 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:42.682304 11300 ts_manager.cc:194] Registered new tserver with Master: 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:42.683920 11300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34874
I20260709 13:14:42.685261 10252 webserver.cc:533] Webserver started at http://127.10.3.3:33563/ using document root <none> and password file <none>
I20260709 13:14:42.685762 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:42.686002 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:42.686283 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:42.687453 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/instance:
uuid: "9e9c2603c0bd4d25b713649b3890c46b"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.691709 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:14:42.694715 11513 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.695497 10252 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:42.695775 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root
uuid: "9e9c2603c0bd4d25b713649b3890c46b"
format_stamp: "Formatted at 2026-07-09 13:14:42 on dist-test-slave-gvjr"
I20260709 13:14:42.696040 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:42.709385 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:42.710526 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:42.711994 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:42.714120 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:42.714349 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.714589 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:42.714759 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:42.753525 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.3:46781
I20260709 13:14:42.753738 11575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.3:46781 every 8 connection(s)
I20260709 13:14:42.767256 11576 heartbeater.cc:344] Connected to a master server at 127.10.3.62:34815
I20260709 13:14:42.767654 11576 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:42.768461 11576 heartbeater.cc:507] Master 127.10.3.62:34815 requested a full tablet report, sending...
I20260709 13:14:42.770412 11300 ts_manager.cc:194] Registered new tserver with Master: 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:42.771368 10252 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014570012s
I20260709 13:14:42.771966 11300 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34878
I20260709 13:14:42.796502 11300 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34892:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:42.802677 11300 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:42.999841 11392 tablet_service.cc:1511] Processing CreateTablet for tablet 9b37b6683af04823b0814ba56c4ea803 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:43.001291 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b37b6683af04823b0814ba56c4ea803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.003094 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02abb4f52449679a2415346f354340 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:43.004550 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02abb4f52449679a2415346f354340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.005188 11390 tablet_service.cc:1511] Processing CreateTablet for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:43.006548 11390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.008054 11389 tablet_service.cc:1511] Processing CreateTablet for tablet 43c1c5858a0f4571a21e2d097910aa44 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:43.009357 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c1c5858a0f4571a21e2d097910aa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.012140 11388 tablet_service.cc:1511] Processing CreateTablet for tablet 08ebf4312af74188830a07342d0dc119 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:43.013456 11388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ebf4312af74188830a07342d0dc119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.013701 11467 tablet_service.cc:1511] Processing CreateTablet for tablet 9b37b6683af04823b0814ba56c4ea803 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:43.014906 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b37b6683af04823b0814ba56c4ea803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.016028 11463 tablet_service.cc:1511] Processing CreateTablet for tablet 08ebf4312af74188830a07342d0dc119 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:43.016527 11387 tablet_service.cc:1511] Processing CreateTablet for tablet 778170f25cb24e08b57cbbc852e8a007 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:43.017367 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ebf4312af74188830a07342d0dc119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.017839 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778170f25cb24e08b57cbbc852e8a007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.020098 11386 tablet_service.cc:1511] Processing CreateTablet for tablet b10b16798ae24794b186d5daa625c6d1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:43.021226 11465 tablet_service.cc:1511] Processing CreateTablet for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:43.022521 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.023627 11385 tablet_service.cc:1511] Processing CreateTablet for tablet e106bf87136140b4bebad713d83d02fe (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:43.024976 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e106bf87136140b4bebad713d83d02fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.026402 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10b16798ae24794b186d5daa625c6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.028159 11464 tablet_service.cc:1511] Processing CreateTablet for tablet 43c1c5858a0f4571a21e2d097910aa44 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:43.028940 11462 tablet_service.cc:1511] Processing CreateTablet for tablet 778170f25cb24e08b57cbbc852e8a007 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:43.029518 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c1c5858a0f4571a21e2d097910aa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.030225 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778170f25cb24e08b57cbbc852e8a007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.031759 11384 tablet_service.cc:1511] Processing CreateTablet for tablet eeb668f66ce344e9acdf8834c2b1d3a3 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:43.033035 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb668f66ce344e9acdf8834c2b1d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.035283 11458 tablet_service.cc:1511] Processing CreateTablet for tablet 38b39fd56a8d45f7b56a9ac114b66d2b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:43.036032 11459 tablet_service.cc:1511] Processing CreateTablet for tablet eeb668f66ce344e9acdf8834c2b1d3a3 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:43.036640 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b39fd56a8d45f7b56a9ac114b66d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.037334 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb668f66ce344e9acdf8834c2b1d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.040040 11460 tablet_service.cc:1511] Processing CreateTablet for tablet e106bf87136140b4bebad713d83d02fe (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:43.041586 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e106bf87136140b4bebad713d83d02fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.043882 11461 tablet_service.cc:1511] Processing CreateTablet for tablet b10b16798ae24794b186d5daa625c6d1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:43.045477 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10b16798ae24794b186d5daa625c6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.049111 11383 tablet_service.cc:1511] Processing CreateTablet for tablet 38b39fd56a8d45f7b56a9ac114b66d2b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:43.050758 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b39fd56a8d45f7b56a9ac114b66d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.020098 11466 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02abb4f52449679a2415346f354340 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:43.053344 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02abb4f52449679a2415346f354340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.152868 11463 tablet_service.cc:1511] Processing CreateTablet for tablet 930d965be7eb49448fa23456beb9925a (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:43.154788 11591 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:43.156538 11458 tablet_service.cc:1511] Processing CreateTablet for tablet a6454d815ae44cdda2fe8d013fca9b4b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:43.157809 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6454d815ae44cdda2fe8d013fca9b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.159006 11541 tablet_service.cc:1511] Processing CreateTablet for tablet 9b37b6683af04823b0814ba56c4ea803 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:43.160228 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b37b6683af04823b0814ba56c4ea803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.162055 11467 tablet_service.cc:1511] Processing CreateTablet for tablet dba44ee610b54495a5f1cfea0a876524 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:43.164065 11540 tablet_service.cc:1511] Processing CreateTablet for tablet 6d02abb4f52449679a2415346f354340 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:43.164649 11462 tablet_service.cc:1511] Processing CreateTablet for tablet b150719d534f439aa33eb70af414a5d0 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:43.165323 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02abb4f52449679a2415346f354340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.167999 11539 tablet_service.cc:1511] Processing CreateTablet for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:43.170284 11538 tablet_service.cc:1511] Processing CreateTablet for tablet 43c1c5858a0f4571a21e2d097910aa44 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:43.172448 11537 tablet_service.cc:1511] Processing CreateTablet for tablet 08ebf4312af74188830a07342d0dc119 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:43.173738 11387 tablet_service.cc:1511] Processing CreateTablet for tablet dba44ee610b54495a5f1cfea0a876524 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:43.174934 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba44ee610b54495a5f1cfea0a876524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.179330 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930d965be7eb49448fa23456beb9925a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.176147 11390 tablet_service.cc:1511] Processing CreateTablet for tablet b150719d534f439aa33eb70af414a5d0 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:43.183877 11390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b150719d534f439aa33eb70af414a5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.196745 11533 tablet_service.cc:1511] Processing CreateTablet for tablet eeb668f66ce344e9acdf8834c2b1d3a3 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:43.199769 11532 tablet_service.cc:1511] Processing CreateTablet for tablet 38b39fd56a8d45f7b56a9ac114b66d2b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:43.200964 11392 tablet_service.cc:1511] Processing CreateTablet for tablet 48129c3807a242fca6439f00aefd9b22 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:43.186122 11459 tablet_service.cc:1511] Processing CreateTablet for tablet 44539667cbc04657b801ff659dee5bde (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:43.176368 11535 tablet_service.cc:1511] Processing CreateTablet for tablet b10b16798ae24794b186d5daa625c6d1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:43.190618 11386 tablet_service.cc:1511] Processing CreateTablet for tablet 44539667cbc04657b801ff659dee5bde (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:43.226679 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48129c3807a242fca6439f00aefd9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.192400 11461 tablet_service.cc:1511] Processing CreateTablet for tablet b66bfe782a6d489fa35dc079b8c4d0a5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:43.192986 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.227444 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44539667cbc04657b801ff659dee5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.230007 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b39fd56a8d45f7b56a9ac114b66d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.192667 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9e663b46c5417b9a0e42fdc7b70969 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:43.231803 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9e663b46c5417b9a0e42fdc7b70969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.196696 11460 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9e663b46c5417b9a0e42fdc7b70969 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:43.184108 11592 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:43.235697 11537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ebf4312af74188830a07342d0dc119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.204231 11389 tablet_service.cc:1511] Processing CreateTablet for tablet 255638a15529487e8ade55aaaa8455d5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:43.184460 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b150719d534f439aa33eb70af414a5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.239723 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255638a15529487e8ade55aaaa8455d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.177584 11534 tablet_service.cc:1511] Processing CreateTablet for tablet e106bf87136140b4bebad713d83d02fe (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:43.240383 11463 tablet_service.cc:1511] Processing CreateTablet for tablet 255638a15529487e8ade55aaaa8455d5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:43.240908 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e106bf87136140b4bebad713d83d02fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.209153 11465 tablet_service.cc:1511] Processing CreateTablet for tablet a3205d9e6dce4aad891fa90943f21879 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:43.212572 11464 tablet_service.cc:1511] Processing CreateTablet for tablet 48129c3807a242fca6439f00aefd9b22 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:43.213399 11591 tablet_bootstrap.cc:654] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.242022 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c1c5858a0f4571a21e2d097910aa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.242848 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10b16798ae24794b186d5daa625c6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.214555 11383 tablet_service.cc:1511] Processing CreateTablet for tablet b66bfe782a6d489fa35dc079b8c4d0a5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:43.245334 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66bfe782a6d489fa35dc079b8c4d0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.172477 11388 tablet_service.cc:1511] Processing CreateTablet for tablet 930d965be7eb49448fa23456beb9925a (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:43.247607 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb668f66ce344e9acdf8834c2b1d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.248967 11388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930d965be7eb49448fa23456beb9925a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.174644 11536 tablet_service.cc:1511] Processing CreateTablet for tablet 778170f25cb24e08b57cbbc852e8a007 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:43.239776 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9e663b46c5417b9a0e42fdc7b70969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.210671 11384 tablet_service.cc:1511] Processing CreateTablet for tablet a3205d9e6dce4aad891fa90943f21879 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:43.251896 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3205d9e6dce4aad891fa90943f21879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.255507 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48129c3807a242fca6439f00aefd9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.247439 11592 tablet_bootstrap.cc:654] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.259298 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44539667cbc04657b801ff659dee5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.260046 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3205d9e6dce4aad891fa90943f21879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.261298 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba44ee610b54495a5f1cfea0a876524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.267288 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255638a15529487e8ade55aaaa8455d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.271828 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66bfe782a6d489fa35dc079b8c4d0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.277817 11458 tablet_service.cc:1511] Processing CreateTablet for tablet ffe5740695024e83afc5069cfb41118c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:14:43.279338 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778170f25cb24e08b57cbbc852e8a007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.292604 11392 tablet_service.cc:1511] Processing CreateTablet for tablet ffe5740695024e83afc5069cfb41118c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:14:43.299134 11591 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:43.301414 11591 ts_tablet_manager.cc:1403] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.005s
I20260709 13:14:43.304361 11591 raft_consensus.cc:359] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.306099 11591 raft_consensus.cc:385] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:43.306481 11591 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:43.305811 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe5740695024e83afc5069cfb41118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.311272 11591 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.313160 11383 tablet_service.cc:1511] Processing CreateTablet for tablet a11e4197bc8843c69871c3f11dc98474 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:14:43.320799 11386 tablet_service.cc:1511] Processing CreateTablet for tablet fe707c46816d4d84bf4ead0d04de2b26 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:43.172474 11385 tablet_service.cc:1511] Processing CreateTablet for tablet a6454d815ae44cdda2fe8d013fca9b4b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:43.327055 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe5740695024e83afc5069cfb41118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.331627 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a11e4197bc8843c69871c3f11dc98474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.345750 11466 tablet_service.cc:1511] Processing CreateTablet for tablet a11e4197bc8843c69871c3f11dc98474 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 13:14:43.350665 11286 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 90169.9us
I20260709 13:14:43.353452 11390 tablet_service.cc:1511] Processing CreateTablet for tablet cfd1f78aa1d843c28021b808cf5fd960 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:43.356496 11387 tablet_service.cc:1511] Processing CreateTablet for tablet 79d2059876e34ff89bf967e96ab7bf09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:43.359548 11384 tablet_service.cc:1511] Processing CreateTablet for tablet 56b606c6942f42cf914bcabc7e2bc502 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:43.361876 11388 tablet_service.cc:1511] Processing CreateTablet for tablet b2eeda82ae3942d6bf0db873fa350bdc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:43.364557 11460 tablet_service.cc:1511] Processing CreateTablet for tablet fe707c46816d4d84bf4ead0d04de2b26 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:43.366557 11389 tablet_service.cc:1511] Processing CreateTablet for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:43.380858 11390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd1f78aa1d843c28021b808cf5fd960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.385614 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d2059876e34ff89bf967e96ab7bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.385761 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a11e4197bc8843c69871c3f11dc98474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.386855 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd40007d144575bd9e785e609a62df (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:43.388093 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd40007d144575bd9e785e609a62df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.388737 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b606c6942f42cf914bcabc7e2bc502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.389273 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe707c46816d4d84bf4ead0d04de2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.391343 11388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2eeda82ae3942d6bf0db873fa350bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.392262 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe707c46816d4d84bf4ead0d04de2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.397045 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 930d965be7eb49448fa23456beb9925a (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:43.398505 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930d965be7eb49448fa23456beb9925a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:43.404484 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:43.405095 11592 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:43.405891 11502 heartbeater.cc:499] Master 127.10.3.62:34815 was elected leader, sending a full tablet report...
I20260709 13:14:43.366557 11465 tablet_service.cc:1511] Processing CreateTablet for tablet b2eeda82ae3942d6bf0db873fa350bdc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:43.408048 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2eeda82ae3942d6bf0db873fa350bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.412029 11459 tablet_service.cc:1511] Processing CreateTablet for tablet 79d2059876e34ff89bf967e96ab7bf09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:43.413302 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d2059876e34ff89bf967e96ab7bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.416579 11536 tablet_service.cc:1511] Processing CreateTablet for tablet a6454d815ae44cdda2fe8d013fca9b4b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:43.417574 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.421264 11463 tablet_service.cc:1511] Processing CreateTablet for tablet cfd1f78aa1d843c28021b808cf5fd960 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:43.422509 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd1f78aa1d843c28021b808cf5fd960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.426545 11541 tablet_service.cc:1511] Processing CreateTablet for tablet dba44ee610b54495a5f1cfea0a876524 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:43.430737 11596 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:43.431094 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6454d815ae44cdda2fe8d013fca9b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.431303 11538 tablet_service.cc:1511] Processing CreateTablet for tablet b150719d534f439aa33eb70af414a5d0 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:43.430773 11591 ts_tablet_manager.cc:1434] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.129s	user 0.038s	sys 0.030s
I20260709 13:14:43.433102 11591 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:43.450073 11464 tablet_service.cc:1511] Processing CreateTablet for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:43.435571 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6454d815ae44cdda2fe8d013fca9b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.405622 11592 ts_tablet_manager.cc:1403] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.222s	user 0.015s	sys 0.020s
I20260709 13:14:43.453141 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b150719d534f439aa33eb70af414a5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.453678 11592 raft_consensus.cc:359] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.454419 11592 raft_consensus.cc:385] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:43.459681 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.454731 11592 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:43.468926 11539 tablet_service.cc:1511] Processing CreateTablet for tablet 44539667cbc04657b801ff659dee5bde (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:43.469413 11535 tablet_service.cc:1511] Processing CreateTablet for tablet b66bfe782a6d489fa35dc079b8c4d0a5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:43.471374 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba44ee610b54495a5f1cfea0a876524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.474040 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66bfe782a6d489fa35dc079b8c4d0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.474622 11591 tablet_bootstrap.cc:654] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.475616 11533 tablet_service.cc:1511] Processing CreateTablet for tablet a3205d9e6dce4aad891fa90943f21879 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:43.476900 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3205d9e6dce4aad891fa90943f21879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.480108 11540 tablet_service.cc:1511] Processing CreateTablet for tablet 48129c3807a242fca6439f00aefd9b22 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:43.481444 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48129c3807a242fca6439f00aefd9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.482002 11592 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.486078 11458 tablet_service.cc:1511] Processing CreateTablet for tablet 56b606c6942f42cf914bcabc7e2bc502 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:43.487679 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b606c6942f42cf914bcabc7e2bc502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.499032 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44539667cbc04657b801ff659dee5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.501255 11534 tablet_service.cc:1511] Processing CreateTablet for tablet 5e9e663b46c5417b9a0e42fdc7b70969 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:43.509013 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9e663b46c5417b9a0e42fdc7b70969. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:43.527480 11284 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54030.3us
I20260709 13:14:43.534205 11532 tablet_service.cc:1511] Processing CreateTablet for tablet 255638a15529487e8ade55aaaa8455d5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:43.567801 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255638a15529487e8ade55aaaa8455d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.544503 11596 tablet_bootstrap.cc:654] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.544291 11389 tablet_service.cc:1511] Processing CreateTablet for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:43.568977 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.549754 11462 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd40007d144575bd9e785e609a62df (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:43.552983 11392 tablet_service.cc:1511] Processing CreateTablet for tablet e8e0fc54e8b645deb82c6013c5d5c7ea (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:43.554154 11386 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8ed42aaa39460bbea1f8deeccd0adc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:43.542357 11383 tablet_service.cc:1511] Processing CreateTablet for tablet 902c85625c4647d481fa8ef07be336f4 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:43.563146 11390 tablet_service.cc:1511] Processing CreateTablet for tablet f52a477150c345c8ba80211d1a49b41c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:43.542402 11467 tablet_service.cc:1511] Processing CreateTablet for tablet 902c85625c4647d481fa8ef07be336f4 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:43.572712 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8ed42aaa39460bbea1f8deeccd0adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.576118 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd40007d144575bd9e785e609a62df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.588853 11388 tablet_service.cc:1511] Processing CreateTablet for tablet ff56d6db2ff44eb19771b9166c042077 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:43.579671 11538 tablet_service.cc:1511] Processing CreateTablet for tablet ffe5740695024e83afc5069cfb41118c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:14:43.595536 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c85625c4647d481fa8ef07be336f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.596338 11592 ts_tablet_manager.cc:1434] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.145s	user 0.013s	sys 0.011s
I20260709 13:14:43.600747 11427 heartbeater.cc:499] Master 127.10.3.62:34815 was elected leader, sending a full tablet report...
I20260709 13:14:43.601218 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e0fc54e8b645deb82c6013c5d5c7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.603474 11464 tablet_service.cc:1511] Processing CreateTablet for tablet e8e0fc54e8b645deb82c6013c5d5c7ea (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:43.604930 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e0fc54e8b645deb82c6013c5d5c7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.610663 11461 tablet_service.cc:1511] Processing CreateTablet for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:43.611950 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.614070 11460 tablet_service.cc:1511] Processing CreateTablet for tablet 74bea11400ce497283dceb7d21b6e728 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:43.623701 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bea11400ce497283dceb7d21b6e728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.627257 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe5740695024e83afc5069cfb41118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.630026 11537 tablet_service.cc:1511] Processing CreateTablet for tablet fe707c46816d4d84bf4ead0d04de2b26 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:43.634186 11592 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:43.634146 11596 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:43.635118 11596 ts_tablet_manager.cc:1403] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.205s	user 0.008s	sys 0.013s
I20260709 13:14:43.624680 11534 tablet_service.cc:1511] Processing CreateTablet for tablet a11e4197bc8843c69871c3f11dc98474 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:14:43.627295 11384 tablet_service.cc:1511] Processing CreateTablet for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:43.635655 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4eb2d16aa247998d0927eb34dedfa8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:43.636147 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a11e4197bc8843c69871c3f11dc98474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.636189 11390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52a477150c345c8ba80211d1a49b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.638680 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c85625c4647d481fa8ef07be336f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.637657 11389 tablet_service.cc:1511] Processing CreateTablet for tablet 5b22c9e294a846159d4831b92f692943 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:43.639621 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b22c9e294a846159d4831b92f692943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.641445 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4eb2d16aa247998d0927eb34dedfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.647459 11598 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.647945 11598 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.650074 11591 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:43.650570 11591 ts_tablet_manager.cc:1403] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.218s	user 0.024s	sys 0.011s
I20260709 13:14:43.651583 11536 tablet_service.cc:1511] Processing CreateTablet for tablet b2eeda82ae3942d6bf0db873fa350bdc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:43.652271 11539 tablet_service.cc:1511] Processing CreateTablet for tablet cfd1f78aa1d843c28021b808cf5fd960 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:43.673048 11595 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.673549 11595 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.678001 11595 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:43.669296 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.671629 11537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe707c46816d4d84bf4ead0d04de2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.655378 11596 raft_consensus.cc:359] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.695577 11596 raft_consensus.cc:385] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:43.695924 11596 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:43.696365 11592 tablet_bootstrap.cc:654] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.696794 11596 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.707615 11591 raft_consensus.cc:359] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.708382 11591 raft_consensus.cc:385] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:43.708675 11591 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:43.709419 11591 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.691788 11598 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:43.714334 11535 tablet_service.cc:1511] Processing CreateTablet for tablet 7abd40007d144575bd9e785e609a62df (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:43.715364 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2eeda82ae3942d6bf0db873fa350bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.720083 11388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff56d6db2ff44eb19771b9166c042077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.725749 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd1f78aa1d843c28021b808cf5fd960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.745571 11386 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff2b5fd08d4882a5355cf7758991cf (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:43.747769 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd40007d144575bd9e785e609a62df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.749209 11385 tablet_service.cc:1511] Processing CreateTablet for tablet 840a1b524ac54acdb12289fa1aafbd17 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:14:43.767378 11591 ts_tablet_manager.cc:1434] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.116s	user 0.008s	sys 0.000s
I20260709 13:14:43.754823 11387 tablet_service.cc:1511] Processing CreateTablet for tablet 74bea11400ce497283dceb7d21b6e728 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:43.772050 11591 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:43.774633 11576 heartbeater.cc:499] Master 127.10.3.62:34815 was elected leader, sending a full tablet report...
I20260709 13:14:43.775375 11463 tablet_service.cc:1511] Processing CreateTablet for tablet f52a477150c345c8ba80211d1a49b41c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:43.776682 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52a477150c345c8ba80211d1a49b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.777257 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff2b5fd08d4882a5355cf7758991cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.781740 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bea11400ce497283dceb7d21b6e728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.763574 11466 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8ed42aaa39460bbea1f8deeccd0adc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:43.791235 11606 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.791882 11606 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.758531 11392 tablet_service.cc:1511] Processing CreateTablet for tablet 6bea8de48c2942d99a1477b5b427be03 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:43.811966 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8ed42aaa39460bbea1f8deeccd0adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.813467 11596 ts_tablet_manager.cc:1434] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.174s	user 0.027s	sys 0.001s
I20260709 13:14:43.814548 11596 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:43.824755 11538 tablet_service.cc:1511] Processing CreateTablet for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:43.826063 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.836243 11464 tablet_service.cc:1511] Processing CreateTablet for tablet ff56d6db2ff44eb19771b9166c042077 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:43.836828 11460 tablet_service.cc:1511] Processing CreateTablet for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:43.837561 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff56d6db2ff44eb19771b9166c042077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.810896 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840a1b524ac54acdb12289fa1aafbd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.838563 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.788373 11540 tablet_service.cc:1511] Processing CreateTablet for tablet 79d2059876e34ff89bf967e96ab7bf09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:43.840939 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bea8de48c2942d99a1477b5b427be03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.842167 11467 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4eb2d16aa247998d0927eb34dedfa8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:43.843739 11465 tablet_service.cc:1511] Processing CreateTablet for tablet 5b22c9e294a846159d4831b92f692943 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:43.877137 11591 tablet_bootstrap.cc:654] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:43.898165 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b22c9e294a846159d4831b92f692943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.907413 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4eb2d16aa247998d0927eb34dedfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.913713 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:43.915540 11365 leader_election.cc:343] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:43.895218 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d2059876e34ff89bf967e96ab7bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.925426 11458 tablet_service.cc:1511] Processing CreateTablet for tablet 6bea8de48c2942d99a1477b5b427be03 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:43.927038 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:43.931505 11367 leader_election.cc:343] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:43.932108 11367 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:43.933063 11598 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:43.933925 11532 tablet_service.cc:1511] Processing CreateTablet for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:43.936324 11462 tablet_service.cc:1511] Processing CreateTablet for tablet 840a1b524ac54acdb12289fa1aafbd17 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:14:43.939074 11592 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:43.939553 11592 ts_tablet_manager.cc:1403] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.306s	user 0.005s	sys 0.014s
I20260709 13:14:43.940712 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 56b606c6942f42cf914bcabc7e2bc502 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:43.941995 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b606c6942f42cf914bcabc7e2bc502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.942451 11592 raft_consensus.cc:359] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.859025 11541 tablet_service.cc:1511] Processing CreateTablet for tablet 902c85625c4647d481fa8ef07be336f4 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:43.943221 11592 raft_consensus.cc:385] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:43.943581 11592 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:43.944023 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c85625c4647d481fa8ef07be336f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.944461 11592 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:43.926841 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bea8de48c2942d99a1477b5b427be03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.875897 11606 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:43.946924 11592 ts_tablet_manager.cc:1434] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:14:43.958801 11592 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:43.960449 11386 tablet_service.cc:1511] Processing CreateTablet for tablet 59af340b26264692af5490466dd69b78 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:14:43.961750 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af340b26264692af5490466dd69b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.962934 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840a1b524ac54acdb12289fa1aafbd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.967130 11461 tablet_service.cc:1511] Processing CreateTablet for tablet 59af340b26264692af5490466dd69b78 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:14:43.981770 11535 tablet_service.cc:1511] Processing CreateTablet for tablet 5e8ed42aaa39460bbea1f8deeccd0adc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:43.992522 11539 tablet_service.cc:1511] Processing CreateTablet for tablet e8e0fc54e8b645deb82c6013c5d5c7ea (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:43.999492 11595 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.000259 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.000018 11595 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.002223 11595 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:44.002612 11442 leader_election.cc:343] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.004657 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.005187 11462 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4de37bdc334587a90c03b40bfd6c5f (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 13:14:44.006069 11442 leader_election.cc:343] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.006744 11387 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4de37bdc334587a90c03b40bfd6c5f (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:14:44.007975 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4de37bdc334587a90c03b40bfd6c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.011229 11383 tablet_service.cc:1511] Processing CreateTablet for tablet c3fd80658464474dbe463b1943becc75 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:44.011360 11459 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff2b5fd08d4882a5355cf7758991cf (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:43.971719 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af340b26264692af5490466dd69b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.015386 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4de37bdc334587a90c03b40bfd6c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.027210 11596 tablet_bootstrap.cc:700] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent opening tablet: real 0.104s	user 0.002s	sys 0.003s
I20260709 13:14:44.027879 11596 tablet_bootstrap.cc:654] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.030738 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff2b5fd08d4882a5355cf7758991cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:43.975636 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.065192 11534 tablet_service.cc:1511] Processing CreateTablet for tablet f52a477150c345c8ba80211d1a49b41c (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:44.058465 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e0fc54e8b645deb82c6013c5d5c7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.072216 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:44.073184 11591 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.073717 11591 ts_tablet_manager.cc:1403] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.302s	user 0.015s	sys 0.008s
I20260709 13:14:44.076996 11592 tablet_bootstrap.cc:654] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.079175 11591 raft_consensus.cc:359] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.079960 11591 raft_consensus.cc:385] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.080302 11591 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
W20260709 13:14:44.081329 11514 leader_election.cc:343] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.082795 11389 tablet_service.cc:1511] Processing CreateTablet for tablet b61ac66d97f84c1186c6e2b1d79477b2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:44.085757 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8ed42aaa39460bbea1f8deeccd0adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.087692 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.088389 11402 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.089680 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.089774 11536 tablet_service.cc:1511] Processing CreateTablet for tablet ff56d6db2ff44eb19771b9166c042077 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:44.090538 11595 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.090891 11595 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.091252 11595 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.095557 11466 tablet_service.cc:1511] Processing CreateTablet for tablet c3fd80658464474dbe463b1943becc75 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:44.098251 11595 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.099421 11463 tablet_service.cc:1511] Processing CreateTablet for tablet b61ac66d97f84c1186c6e2b1d79477b2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:44.101719 11463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ac66d97f84c1186c6e2b1d79477b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.108474 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3fd80658464474dbe463b1943becc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.108268 11591 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.108811 11385 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba703fc47843af924d4e45f12db16b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:44.114133 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.115218 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff56d6db2ff44eb19771b9166c042077. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:44.115754 11516 leader_election.cc:343] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.109459 11596 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.116223 11516 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:44.116851 11596 ts_tablet_manager.cc:1403] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.303s	user 0.007s	sys 0.013s
I20260709 13:14:44.117456 11606 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.115381 11387 tablet_service.cc:1511] Processing CreateTablet for tablet aa999fcc28454bc88f93770d07ff72e2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:44.107514 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.127542 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52a477150c345c8ba80211d1a49b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.134225 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3fd80658464474dbe463b1943becc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.134678 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:44.135294 11402 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.139791 11467 tablet_service.cc:1511] Processing CreateTablet for tablet aa999fcc28454bc88f93770d07ff72e2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:44.140190 11384 tablet_service.cc:1511] Processing CreateTablet for tablet 058fa5ec3edb42c796103c7361c1a8e1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:44.141080 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa999fcc28454bc88f93770d07ff72e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.142004 11465 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba703fc47843af924d4e45f12db16b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:44.143299 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba703fc47843af924d4e45f12db16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.142406 11388 tablet_service.cc:1511] Processing CreateTablet for tablet 13474bcfb1f1443f82e90db3445bc880 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:44.145102 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ac66d97f84c1186c6e2b1d79477b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.146507 11595 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.154704 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:44.156903 11540 tablet_service.cc:1511] Processing CreateTablet for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:44.158221 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:44.158141 11442 leader_election.cc:343] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:44.163546 11442 leader_election.cc:343] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.163990 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.164503 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba703fc47843af924d4e45f12db16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.165078 11595 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.160326 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 95e3d59427ee46f989532eb3cdc311cc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:44.166491 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e3d59427ee46f989532eb3cdc311cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.167335 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058fa5ec3edb42c796103c7361c1a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.168972 11388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13474bcfb1f1443f82e90db3445bc880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.173113 11591 ts_tablet_manager.cc:1434] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.012s
I20260709 13:14:44.180227 11392 tablet_service.cc:1511] Processing CreateTablet for tablet 1f946b7973fe482681135635493ebd48 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:44.181120 11592 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.181610 11592 ts_tablet_manager.cc:1403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.223s	user 0.009s	sys 0.006s
I20260709 13:14:44.185185 11591 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.185911 11391 tablet_service.cc:1511] Processing CreateTablet for tablet 4f615bf163b9483d8209eb6c8fedca15 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:44.188549 11461 tablet_service.cc:1511] Processing CreateTablet for tablet 13474bcfb1f1443f82e90db3445bc880 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:44.189815 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13474bcfb1f1443f82e90db3445bc880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.188549 11462 tablet_service.cc:1511] Processing CreateTablet for tablet 058fa5ec3edb42c796103c7361c1a8e1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:44.191380 11541 tablet_service.cc:1511] Processing CreateTablet for tablet 1c4eb2d16aa247998d0927eb34dedfa8 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:44.192751 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4eb2d16aa247998d0927eb34dedfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.193106 11537 tablet_service.cc:1511] Processing CreateTablet for tablet 74bea11400ce497283dceb7d21b6e728 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:44.197264 11592 raft_consensus.cc:359] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.200146 11592 raft_consensus.cc:385] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.200024 11390 tablet_service.cc:1511] Processing CreateTablet for tablet 8e90063696c0437bb8ad2d545f45a9ce (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:44.200547 11592 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.201429 11592 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.204481 11462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058fa5ec3edb42c796103c7361c1a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.208150 11464 tablet_service.cc:1511] Processing CreateTablet for tablet 8e90063696c0437bb8ad2d545f45a9ce (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:44.209517 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e90063696c0437bb8ad2d545f45a9ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.190543 11458 tablet_service.cc:1511] Processing CreateTablet for tablet 95e3d59427ee46f989532eb3cdc311cc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:44.211771 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e3d59427ee46f989532eb3cdc311cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.193276 11538 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff2b5fd08d4882a5355cf7758991cf (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:44.198969 11387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa999fcc28454bc88f93770d07ff72e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.203352 11596 raft_consensus.cc:359] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.215898 11596 raft_consensus.cc:385] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.216231 11596 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.216980 11596 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.219278 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f946b7973fe482681135635493ebd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.219537 11459 tablet_service.cc:1511] Processing CreateTablet for tablet 1f946b7973fe482681135635493ebd48 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:44.220732 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f946b7973fe482681135635493ebd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.215605 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 59af340b26264692af5490466dd69b78 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:14:44.199689 11537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bea11400ce497283dceb7d21b6e728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.223557 11391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f615bf163b9483d8209eb6c8fedca15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.232363 11386 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:14:44.233188 11402 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:44.234390 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.203918 11592 ts_tablet_manager.cc:1434] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 13:14:44.236483 11595 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:44.236963 11595 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:44.237795 11595 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.227957 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff2b5fd08d4882a5355cf7758991cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.243956 11591 tablet_bootstrap.cc:654] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.247506 11390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e90063696c0437bb8ad2d545f45a9ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.248389 11596 ts_tablet_manager.cc:1434] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.131s	user 0.008s	sys 0.002s
I20260709 13:14:44.249337 11596 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.251332 11386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.251854 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af340b26264692af5490466dd69b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.255697 11592 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.261073 11535 tablet_service.cc:1511] Processing CreateTablet for tablet 5b22c9e294a846159d4831b92f692943 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:44.268585 11532 tablet_service.cc:1511] Processing CreateTablet for tablet 6bea8de48c2942d99a1477b5b427be03 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:44.272228 11460 tablet_service.cc:1511] Processing CreateTablet for tablet 4f615bf163b9483d8209eb6c8fedca15 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:44.274188 11460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f615bf163b9483d8209eb6c8fedca15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.279074 11539 tablet_service.cc:1511] Processing CreateTablet for tablet 840a1b524ac54acdb12289fa1aafbd17 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:14:44.262862 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b22c9e294a846159d4831b92f692943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.303802 11389 tablet_service.cc:1511] Processing CreateTablet for tablet b5de422ef4884189bea49cec7b7c2ec1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:14:44.305114 11389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5de422ef4884189bea49cec7b7c2ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.311864 11596 tablet_bootstrap.cc:654] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.312678 11536 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4de37bdc334587a90c03b40bfd6c5f (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:14:44.314602 11466 tablet_service.cc:1511] Processing CreateTablet for tablet b5de422ef4884189bea49cec7b7c2ec1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:14:44.316043 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5de422ef4884189bea49cec7b7c2ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.316000 11383 tablet_service.cc:1511] Processing CreateTablet for tablet 66cb4598d33043fc8eb235647a242814 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:14:44.317292 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cb4598d33043fc8eb235647a242814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.321949 11534 tablet_service.cc:1511] Processing CreateTablet for tablet c3fd80658464474dbe463b1943becc75 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:44.336644 11598 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.337399 11591 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.337152 11598 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.340245 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:44.340615 11466 tablet_service.cc:1511] Processing CreateTablet for tablet 66cb4598d33043fc8eb235647a242814 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 13:14:44.341727 11365 leader_election.cc:343] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.342211 11466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cb4598d33043fc8eb235647a242814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.342525 11598 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.344025 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:44.345398 11367 leader_election.cc:343] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.345803 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.346889 11598 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.347802 11591 ts_tablet_manager.cc:1403] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.163s	user 0.019s	sys 0.021s
I20260709 13:14:44.350754 11591 raft_consensus.cc:359] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.354746 11591 raft_consensus.cc:385] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.355108 11591 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.356118 11591 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.358613 11591 ts_tablet_manager.cc:1434] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:14:44.306015 11461 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:14:44.369988 11606 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.370747 11606 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.373970 11606 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:44.374082 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:44.376820 11516 leader_election.cc:343] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.380234 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:44.381641 11514 leader_election.cc:343] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.382095 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:44.383179 11606 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.384886 11461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.391664 11592 tablet_bootstrap.cc:654] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.398115 11467 tablet_service.cc:1511] Processing CreateTablet for tablet 25be57d1c24d4a43859ce0b5848adc09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:44.400871 11467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be57d1c24d4a43859ce0b5848adc09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.403170 11392 tablet_service.cc:1511] Processing CreateTablet for tablet 4e67029edeb04a569eb2c35676ef78da (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:44.404415 11392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e67029edeb04a569eb2c35676ef78da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.410373 11595 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.410864 11595 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.394469 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4de37bdc334587a90c03b40bfd6c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.412858 11595 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.413184 11541 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43418 (request call id 36) took 1017 ms. Trace:
I20260709 13:14:44.413640 11541 rpcz_store.cc:276] 0709 13:14:43.396245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.190506 (+794261us) service_pool.cc:224] Handling call
0709 13:14:44.192324 (+  1818us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:14:44.192420 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:14:44.388485 (+196065us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:44.412721 (+ 24236us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:44.413122 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:44.414736 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.415349 11551 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.416692 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.418951 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:14:44.419818 11442 leader_election.cc:343] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.419905 11540 tablet_service.cc:1511] Processing CreateTablet for tablet b61ac66d97f84c1186c6e2b1d79477b2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:44.421177 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ac66d97f84c1186c6e2b1d79477b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.422441 11595 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.422874 11595 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.423333 11595 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.402606 11384 tablet_service.cc:1511] Processing CreateTablet for tablet 25be57d1c24d4a43859ce0b5848adc09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:44.425506 11541 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba703fc47843af924d4e45f12db16b (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:44.426258 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be57d1c24d4a43859ce0b5848adc09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.426748 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba703fc47843af924d4e45f12db16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.429873 11465 tablet_service.cc:1511] Processing CreateTablet for tablet 4e67029edeb04a569eb2c35676ef78da (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:44.431146 11465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e67029edeb04a569eb2c35676ef78da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.439424 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840a1b524ac54acdb12289fa1aafbd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.439815 11591 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.441895 11300 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:44.450527 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bea8de48c2942d99a1477b5b427be03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.460665 11592 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
W20260709 13:14:44.454639 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:44.470346 11592 ts_tablet_manager.cc:1403] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.232s	user 0.011s	sys 0.003s
I20260709 13:14:44.453394 11538 tablet_service.cc:1511] Processing CreateTablet for tablet aa999fcc28454bc88f93770d07ff72e2 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:44.471398 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3fd80658464474dbe463b1943becc75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.473299 11592 raft_consensus.cc:359] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.489462 11592 raft_consensus.cc:385] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.475734 11595 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.479774 11596 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.484632 11537 tablet_service.cc:1511] Processing CreateTablet for tablet 95e3d59427ee46f989532eb3cdc311cc (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:44.492017 11596 ts_tablet_manager.cc:1403] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.243s	user 0.020s	sys 0.023s
I20260709 13:14:44.497614 11595 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.499459 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:44.500612 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:44.501147 11551 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:44.503707 11442 leader_election.cc:343] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.509279 11551 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:44.510723 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:44.489876 11592 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.513159 11595 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:44.475265 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa999fcc28454bc88f93770d07ff72e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.514540 11537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e3d59427ee46f989532eb3cdc311cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.513001 11592 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.489106 11591 tablet_bootstrap.cc:654] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.495282 11596 raft_consensus.cc:359] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.528142 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 13474bcfb1f1443f82e90db3445bc880 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:44.531334 11596 raft_consensus.cc:385] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.535502 11536 tablet_service.cc:1511] Processing CreateTablet for tablet 058fa5ec3edb42c796103c7361c1a8e1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:44.535931 11596 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.536953 11596 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.542234 11596 ts_tablet_manager.cc:1434] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.050s	user 0.002s	sys 0.004s
I20260709 13:14:44.519387 11595 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:44.536978 11536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058fa5ec3edb42c796103c7361c1a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.547286 11595 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.550419 11596 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.554342 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13474bcfb1f1443f82e90db3445bc880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.556423 11592 ts_tablet_manager.cc:1434] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.086s	user 0.005s	sys 0.002s
I20260709 13:14:44.557441 11592 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.564744 11596 tablet_bootstrap.cc:654] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.566255 11591 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.566148 11540 tablet_service.cc:1511] Processing CreateTablet for tablet 8e90063696c0437bb8ad2d545f45a9ce (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:44.566764 11591 ts_tablet_manager.cc:1403] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.007s
I20260709 13:14:44.567327 11539 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43418 (request call id 40) took 1010 ms. Trace:
I20260709 13:14:44.570477 11592 tablet_bootstrap.cc:654] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.568390 11539 rpcz_store.cc:276] 0709 13:14:43.556625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.277788 (+721163us) service_pool.cc:224] Handling call
0709 13:14:44.280381 (+  2593us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:14:44.280509 (+   128us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:14:44.473746 (+193237us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:44.549965 (+ 76219us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:44.567260 (+ 17295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:44.570622 11619 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.571234 11619 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.574090 11541 tablet_service.cc:1511] Processing CreateTablet for tablet 1f946b7973fe482681135635493ebd48 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:44.574681 11619 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.575495 11541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f946b7973fe482681135635493ebd48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.576408 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.577909 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.578821 11551 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.580513 11539 tablet_service.cc:1511] Processing CreateTablet for tablet 4f615bf163b9483d8209eb6c8fedca15 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:44.581419 11591 raft_consensus.cc:359] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.582118 11591 raft_consensus.cc:385] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.582065 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:44.582585 11591 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
W20260709 13:14:44.583434 11442 leader_election.cc:343] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.584595 11619 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.585068 11619 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.585526 11619 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.585194 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e90063696c0437bb8ad2d545f45a9ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.601100 11534 tablet_service.cc:1511] Processing CreateTablet for tablet b5de422ef4884189bea49cec7b7c2ec1 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:14:44.602439 11534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5de422ef4884189bea49cec7b7c2ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.603272 11539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f615bf163b9483d8209eb6c8fedca15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.604380 11538 tablet_service.cc:1511] Processing CreateTablet for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:14:44.606082 11538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.616762 11535 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43418 (request call id 37) took 1061 ms. Trace:
I20260709 13:14:44.617254 11535 rpcz_store.cc:276] 0709 13:14:43.555807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.260157 (+704350us) service_pool.cc:224] Handling call
0709 13:14:44.262212 (+  2055us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:14:44.262339 (+   127us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:14:44.549424 (+287085us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:44.616211 (+ 66787us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:44.616702 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:44.619551 11535 tablet_service.cc:1511] Processing CreateTablet for tablet 66cb4598d33043fc8eb235647a242814 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:14:44.621552 11596 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
W20260709 13:14:44.623160 11442 consensus_peers.cc:597] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:44.627282 11535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66cb4598d33043fc8eb235647a242814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.629086 11537 tablet_service.cc:1511] Processing CreateTablet for tablet 25be57d1c24d4a43859ce0b5848adc09 (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:44.630373 11537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be57d1c24d4a43859ce0b5848adc09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.607924 11591 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.633132 11591 ts_tablet_manager.cc:1434] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.000s
I20260709 13:14:44.634143 11591 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.634339 11532 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43418 (request call id 39) took 1078 ms. Trace:
I20260709 13:14:44.634752 11532 rpcz_store.cc:276] 0709 13:14:43.556370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.267679 (+711309us) service_pool.cc:224] Handling call
0709 13:14:44.269465 (+  1786us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:14:44.269546 (+    81us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:14:44.627236 (+357690us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:44.633846 (+  6610us) tablet_replica.cc:206] Creating consensus instance
0709 13:14:44.634278 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:44.636663 11532 tablet_service.cc:1511] Processing CreateTablet for tablet 4e67029edeb04a569eb2c35676ef78da (DEFAULT_TABLE table=test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:44.637867 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e67029edeb04a569eb2c35676ef78da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:44.625748 11619 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.641199 11592 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.641700 11592 ts_tablet_manager.cc:1403] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.025s
I20260709 13:14:44.642915 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:14:44.644462 11442 leader_election.cc:343] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.644402 11592 raft_consensus.cc:359] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.645006 11592 raft_consensus.cc:385] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.644902 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:44.645535 11551 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.647126 11592 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.648038 11592 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.650482 11592 ts_tablet_manager.cc:1434] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:14:44.654582 11551 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:44.655755 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:44.656618 11595 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:44.626708 11596 ts_tablet_manager.cc:1403] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.011s
I20260709 13:14:44.663389 11592 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.666150 11598 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.666672 11598 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.645084 11619 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.669073 11598 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.671008 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:44.674943 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.675815 11551 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:44.676929 11367 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.678068 11598 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.678630 11592 tablet_bootstrap.cc:654] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.682400 11595 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:44.684497 11595 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.685990 11598 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.686470 11596 raft_consensus.cc:359] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.688297 11596 raft_consensus.cc:385] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.688648 11596 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.695518 11596 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.700817 11591 tablet_bootstrap.cc:654] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.702095 11592 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.704859 11299 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:44.706243 11592 ts_tablet_manager.cc:1403] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.043s	user 0.006s	sys 0.005s
I20260709 13:14:44.707634 11596 ts_tablet_manager.cc:1434] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:14:44.708612 11596 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.709298 11592 raft_consensus.cc:359] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.710001 11592 raft_consensus.cc:385] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.710291 11592 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.711031 11592 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.713251 11592 ts_tablet_manager.cc:1434] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:44.714215 11592 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.715143 11596 tablet_bootstrap.cc:654] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.720748 11299 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:44.734429 11591 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.734468 11592 tablet_bootstrap.cc:654] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.735106 11591 ts_tablet_manager.cc:1403] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.101s	user 0.018s	sys 0.008s
I20260709 13:14:44.737586 11591 raft_consensus.cc:359] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.738348 11591 raft_consensus.cc:385] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.738610 11591 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.739400 11592 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.739267 11591 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.739920 11592 ts_tablet_manager.cc:1403] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.001s
I20260709 13:14:44.741125 11596 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.741374 11591 ts_tablet_manager.cc:1434] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:44.741567 11596 ts_tablet_manager.cc:1403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.012s
I20260709 13:14:44.742605 11591 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.742799 11592 raft_consensus.cc:359] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.743541 11592 raft_consensus.cc:385] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.743834 11592 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.744436 11596 raft_consensus.cc:359] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.744709 11592 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.745134 11596 raft_consensus.cc:385] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.745573 11596 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.746433 11596 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.748644 11598 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.749102 11598 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.751151 11598 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.752637 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.752646 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:44.754139 11367 leader_election.cc:343] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.748660 11596 ts_tablet_manager.cc:1434] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
W20260709 13:14:44.755050 11365 leader_election.cc:343] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.755321 11592 ts_tablet_manager.cc:1434] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:14:44.755609 11596 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.755492 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:44.757890 11591 tablet_bootstrap.cc:654] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.759203 11598 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:44.762167 11592 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.762526 11596 tablet_bootstrap.cc:654] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.768254 11596 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.768744 11596 ts_tablet_manager.cc:1403] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:14:44.768846 11592 tablet_bootstrap.cc:654] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.769129 11591 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.769553 11591 ts_tablet_manager.cc:1403] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.003s
I20260709 13:14:44.772055 11596 raft_consensus.cc:359] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.772168 11591 raft_consensus.cc:359] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.772627 11596 raft_consensus.cc:385] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.773036 11591 raft_consensus.cc:385] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.773078 11596 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.773526 11591 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.774048 11592 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.774601 11592 ts_tablet_manager.cc:1403] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:14:44.774078 11596 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.774333 11591 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.776831 11596 ts_tablet_manager.cc:1434] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:14:44.777125 11592 raft_consensus.cc:359] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.777771 11592 raft_consensus.cc:385] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.778023 11592 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.778151 11596 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.780671 11591 ts_tablet_manager.cc:1434] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:14:44.781808 11591 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.785841 11596 tablet_bootstrap.cc:654] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.778736 11592 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.788753 11591 tablet_bootstrap.cc:654] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.790952 11592 ts_tablet_manager.cc:1434] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 13:14:44.791888 11592 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.801440 11591 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.801481 11596 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.802098 11596 ts_tablet_manager.cc:1403] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.011s
I20260709 13:14:44.802098 11591 ts_tablet_manager.cc:1403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:14:44.803580 11595 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.803930 11592 tablet_bootstrap.cc:654] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.804124 11595 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.805166 11591 raft_consensus.cc:359] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.805878 11591 raft_consensus.cc:385] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.806171 11591 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.806377 11595 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.807068 11591 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.808627 11592 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.809125 11592 ts_tablet_manager.cc:1403] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 13:14:44.809206 11591 ts_tablet_manager.cc:1434] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:14:44.808929 11596 raft_consensus.cc:359] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.809626 11596 raft_consensus.cc:385] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.809659 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.810036 11596 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.810389 11402 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.810799 11596 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.811795 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:44.812361 11592 raft_consensus.cc:359] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.813020 11592 raft_consensus.cc:385] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.813058 11632 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.813455 11592 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.813571 11632 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.813853 11591 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.814109 11632 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.814455 11592 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.816448 11592 ts_tablet_manager.cc:1434] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:44.808478 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.819537 11596 ts_tablet_manager.cc:1434] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:14:44.820201 11551 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.820902 11596 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.826807 11591 tablet_bootstrap.cc:654] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.827400 11596 tablet_bootstrap.cc:654] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.830973 11592 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.839264 11592 tablet_bootstrap.cc:654] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.842754 11632 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.846272 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:44.846918 11402 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.856067 11595 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:44.858094 11592 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:44.858798 11592 ts_tablet_manager.cc:1403] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.028s	user 0.005s	sys 0.019s
I20260709 13:14:44.871426 11592 raft_consensus.cc:359] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.872439 11592 raft_consensus.cc:385] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.872829 11592 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.873819 11592 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.875279 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:44.875777 11606 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.876169 11551 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.876390 11606 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.880915 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:44.881695 11400 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:44.883395 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:44.883759 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:44.884466 11477 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:44.885039 11606 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:44.885792 11606 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.886253 11606 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.886713 11606 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.893221 11551 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:44.895107 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:44.896515 11595 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:44.898306 11595 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:44.899335 11595 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.902745 11592 ts_tablet_manager.cc:1434] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:14:44.908911 11632 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.916342 11592 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:44.923249 11402 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:44.923168 11297 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:44.931484 11591 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:44.932026 11591 ts_tablet_manager.cc:1403] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.118s	user 0.053s	sys 0.047s
I20260709 13:14:44.933481 11596 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:44.933981 11596 ts_tablet_manager.cc:1403] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.113s	user 0.026s	sys 0.042s
I20260709 13:14:44.933693 11606 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.934762 11591 raft_consensus.cc:359] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.935497 11591 raft_consensus.cc:385] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.935786 11591 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.936450 11591 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.936771 11596 raft_consensus.cc:359] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.937410 11596 raft_consensus.cc:385] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:44.937705 11596 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:44.938812 11591 ts_tablet_manager.cc:1434] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:44.938378 11596 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.939788 11591 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:44.940517 11596 ts_tablet_manager.cc:1434] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:44.943650 11598 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.944207 11598 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.945094 11596 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:44.943723 11606 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:44.946169 11598 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.946523 11591 tablet_bootstrap.cc:654] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.947343 11592 tablet_bootstrap.cc:654] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:44.948503 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:44.949122 11477 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:44.950307 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:44.951897 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:44.952196 11596 tablet_bootstrap.cc:654] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:44.954181 11367 leader_election.cc:343] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:44.961680 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:44.962525 11477 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.966794 11598 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:44.967324 11598 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.967712 11598 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.980108 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:44.980756 11401 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:44.984004 11595 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.984606 11595 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:44.987538 11595 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:44.990432 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.001520 11402 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:44.991420 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.003293 11551 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:45.005358 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.006475 11632 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.006956 11632 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.007387 11632 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.019295 11619 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:45.033955 11632 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.055011 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.055910 11402 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.035997 11598 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.049283 11595 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.079252 11399 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.000634 11401 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.084039 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.051277 11477 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.085850 11606 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.089396 11598 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.089937 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.091312 11632 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.092268 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.093179 11549 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:45.095206 11367 leader_election.cc:343] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.089926 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.096794 11476 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.101498 11596 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.102003 11596 ts_tablet_manager.cc:1403] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.157s	user 0.063s	sys 0.090s
I20260709 13:14:45.102595 11632 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.104835 11596 raft_consensus.cc:359] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.105520 11596 raft_consensus.cc:385] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.105821 11596 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.111408 11606 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:45.113134 11591 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.112923 11606 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.106642 11596 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.108417 11650 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.114487 11650 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.116230 11476 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:45.116557 11596 ts_tablet_manager.cc:1434] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 13:14:45.118286 11592 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.118232 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.118594 11596 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.118432 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.118885 11592 ts_tablet_manager.cc:1403] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.203s	user 0.049s	sys 0.068s
I20260709 13:14:45.118945 11399 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.119659 11598 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.120978 11516 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.121763 11592 raft_consensus.cc:359] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.122298 11659 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.122560 11650 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.122601 11592 raft_consensus.cc:385] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.122749 11659 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.106400 11549 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.123158 11592 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.123265 11659 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.113726 11591 ts_tablet_manager.cc:1403] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.174s	user 0.052s	sys 0.079s
I20260709 13:14:45.123734 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.123876 11592 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.124471 11477 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.125759 11650 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.125841 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.126868 11632 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.127394 11632 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:45.126314 11650 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.128413 11632 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.126778 11591 raft_consensus.cc:359] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.132179 11591 raft_consensus.cc:385] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.132583 11591 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.133494 11591 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.117188 11402 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.141466 11591 ts_tablet_manager.cc:1434] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:14:45.143028 11591 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.144639 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:45.148943 11516 leader_election.cc:343] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.154911 11592 ts_tablet_manager.cc:1434] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.036s	user 0.011s	sys 0.004s
I20260709 13:14:45.154755 11598 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:45.156597 11592 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.156210 11650 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.157094 11656 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:14:45.158697 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.159555 11598 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.158926 11663 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.160121 11476 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.163085 11596 tablet_bootstrap.cc:654] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.162843 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:45.164654 11663 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.159982 11656 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.167790 11663 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.169448 11656 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.170188 11606 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.171064 11606 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.171240 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.171468 11606 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.173355 11659 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.174539 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:45.176234 11365 leader_election.cc:343] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.178736 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.181020 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:45.184194 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.184803 11402 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.184690 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.185523 11549 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:45.187194 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.188270 11664 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.188680 11664 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.188621 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.189257 11664 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.191190 11632 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:45.193856 11656 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:45.196943 11656 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:14:45.197332 11656 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:45.199908 11592 tablet_bootstrap.cc:654] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.194000 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.183166 11659 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.202008 11476 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.200771 11299 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.205581 11402 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.207118 11516 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.207522 11297 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.208396 11650 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.209431 11591 tablet_bootstrap.cc:654] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.213681 11632 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.214157 11632 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.231863 11650 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:45.244583 11670 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.245407 11670 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.245817 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.246622 11402 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:45.247396 11650 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.248337 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:45.249243 11651 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.249569 11651 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.249830 11651 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.249900 11606 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.251186 11656 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.253311 11656 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.238307 11476 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.257278 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.257619 11651 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.257439 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.260665 11670 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.261195 11632 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.265892 11592 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.266223 11596 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.266706 11596 ts_tablet_manager.cc:1403] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.148s	user 0.024s	sys 0.018s
I20260709 13:14:45.266347 11592 ts_tablet_manager.cc:1403] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.110s	user 0.007s	sys 0.005s
I20260709 13:14:45.266311 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.265789 11664 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.271154 11477 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:45.268102 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.272487 11664 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.273608 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.274425 11550 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.273605 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.266777 11402 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.276082 11651 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.279004 11664 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:45.269287 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.274966 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.281412 11551 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:45.283156 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.283701 11401 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.283893 11592 raft_consensus.cc:359] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.284549 11592 raft_consensus.cc:385] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.284811 11592 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.287487 11592 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:45.287773 11365 leader_election.cc:343] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.287690 11516 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.289166 11670 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.289520 11670 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.289698 11592 ts_tablet_manager.cc:1434] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:14:45.289849 11670 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.290668 11592 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.291014 11401 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.291381 11596 raft_consensus.cc:359] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.292032 11596 raft_consensus.cc:385] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.292111 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.292337 11596 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.292189 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:45.293080 11596 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.294481 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.293453 11606 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.286154 11298 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:45.295702 11516 leader_election.cc:343] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.297148 11632 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.297669 11591 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.299170 11591 ts_tablet_manager.cc:1403] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.156s	user 0.012s	sys 0.006s
I20260709 13:14:45.297716 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.299855 11632 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:45.302752 11596 ts_tablet_manager.cc:1434] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:14:45.303635 11596 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.303160 11550 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:45.303974 11632 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.302207 11477 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.305171 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.306932 11664 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.314205 11591 raft_consensus.cc:359] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.315156 11591 raft_consensus.cc:385] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.315490 11591 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.316059 11477 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.316581 11591 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.317359 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:45.318574 11650 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.319280 11650 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:45.319600 11591 ts_tablet_manager.cc:1434] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:14:45.320175 11650 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.321285 11591 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.308524 11664 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:45.323489 11664 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.324600 11596 tablet_bootstrap.cc:654] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.325294 11670 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.326443 11551 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:45.330303 11670 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.331063 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.331612 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.335248 11401 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:45.338490 11514 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.340044 11651 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.340655 11670 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.341064 11651 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.343022 11651 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.346858 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.347155 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:45.348764 11442 leader_election.cc:343] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:45.349675 11442 leader_election.cc:343] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.350196 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.351140 11651 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.369194 11592 tablet_bootstrap.cc:654] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.371621 11297 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.372963 11596 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.373503 11596 ts_tablet_manager.cc:1403] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.004s
I20260709 13:14:45.372884 11298 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.375573 11299 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.377624 11299 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.378315 11596 raft_consensus.cc:359] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.379015 11596 raft_consensus.cc:385] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.379295 11596 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.379884 11596 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.387380 11670 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.387990 11670 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.391139 11670 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.391116 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.391587 11592 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.391580 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.391825 11401 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.392156 11592 ts_tablet_manager.cc:1403] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.102s	user 0.022s	sys 0.005s
I20260709 13:14:45.393039 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:14:45.393606 11514 leader_election.cc:343] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.393961 11650 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.394395 11650 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.394783 11650 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.395664 11591 tablet_bootstrap.cc:654] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.397432 11596 ts_tablet_manager.cc:1434] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:14:45.398455 11596 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.398341 11592 raft_consensus.cc:359] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.399148 11592 raft_consensus.cc:385] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.399434 11592 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.400080 11592 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.402051 11592 ts_tablet_manager.cc:1434] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:14:45.402881 11592 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.402577 11650 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.404862 11596 tablet_bootstrap.cc:654] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.408578 11592 tablet_bootstrap.cc:654] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.409159 11650 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.410234 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.410656 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.411270 11401 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:45.411525 11514 leader_election.cc:343] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.414537 11592 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.415155 11592 ts_tablet_manager.cc:1403] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.003s
I20260709 13:14:45.415537 11591 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.415997 11591 ts_tablet_manager.cc:1403] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.002s
I20260709 13:14:45.418612 11591 raft_consensus.cc:359] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.418566 11592 raft_consensus.cc:359] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.419559 11591 raft_consensus.cc:385] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.419713 11592 raft_consensus.cc:385] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.419932 11591 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.420138 11592 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.420842 11591 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.421481 11592 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.423184 11591 ts_tablet_manager.cc:1434] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:45.424206 11592 ts_tablet_manager.cc:1434] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:14:45.425500 11401 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:45.425736 11591 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.426663 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:45.427464 11592 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.427773 11650 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.428258 11650 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:45.429046 11650 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.432163 11664 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.432752 11664 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.435212 11664 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.438047 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:45.448860 11367 leader_election.cc:343] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.451691 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:45.454501 11365 leader_election.cc:343] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.454943 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.456013 11651 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.457901 11664 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.459733 11592 tablet_bootstrap.cc:654] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.468998 11596 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.469617 11596 ts_tablet_manager.cc:1403] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.013s
I20260709 13:14:45.473338 11596 raft_consensus.cc:359] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.474291 11596 raft_consensus.cc:385] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.474648 11596 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
W20260709 13:14:45.485067 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:45.487748 11651 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.494095 11297 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.496845 11596 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.497102 11650 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.495880 11592 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.501368 11592 ts_tablet_manager.cc:1403] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.027s
I20260709 13:14:45.496126 11591 tablet_bootstrap.cc:654] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.504858 11592 raft_consensus.cc:359] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.505739 11592 raft_consensus.cc:385] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.506065 11592 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.506587 11596 ts_tablet_manager.cc:1434] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.005s
I20260709 13:14:45.507506 11596 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.508850 11592 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.513410 11592 ts_tablet_manager.cc:1434] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:14:45.514605 11592 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.524581 11591 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.524992 11596 tablet_bootstrap.cc:654] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.526394 11592 tablet_bootstrap.cc:654] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.529484 11591 ts_tablet_manager.cc:1403] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.104s	user 0.023s	sys 0.000s
I20260709 13:14:45.532217 11591 raft_consensus.cc:359] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.533867 11591 raft_consensus.cc:385] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.534919 11591 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.535910 11591 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.538790 11591 ts_tablet_manager.cc:1434] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:14:45.540292 11591 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.543695 11596 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.541244 11592 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.544149 11596 ts_tablet_manager.cc:1403] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.006s
I20260709 13:14:45.544225 11592 ts_tablet_manager.cc:1403] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.008s
I20260709 13:14:45.545295 11650 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.546831 11596 raft_consensus.cc:359] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.547550 11596 raft_consensus.cc:385] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.547863 11596 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.548573 11596 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.550509 11596 ts_tablet_manager.cc:1434] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:45.550876 11591 tablet_bootstrap.cc:654] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.551092 11619 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.552724 11592 raft_consensus.cc:359] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.553406 11592 raft_consensus.cc:385] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.553694 11592 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.551432 11596 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.554354 11592 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.555848 11592 ts_tablet_manager.cc:1434] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:14:45.556547 11592 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.565145 11596 tablet_bootstrap.cc:654] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.566601 11619 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:45.571851 11592 tablet_bootstrap.cc:654] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.588706 11591 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.589360 11591 ts_tablet_manager.cc:1403] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.013s
I20260709 13:14:45.592741 11591 raft_consensus.cc:359] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.593684 11591 raft_consensus.cc:385] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.594009 11591 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.595024 11596 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.594777 11591 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.595443 11596 ts_tablet_manager.cc:1403] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.013s
I20260709 13:14:45.596975 11591 ts_tablet_manager.cc:1434] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:45.598318 11591 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.598017 11596 raft_consensus.cc:359] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.598688 11596 raft_consensus.cc:385] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.598964 11596 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.599596 11596 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.601449 11596 ts_tablet_manager.cc:1434] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:14:45.602279 11596 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.604831 11592 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.605345 11592 ts_tablet_manager.cc:1403] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.003s
I20260709 13:14:45.607249 11591 tablet_bootstrap.cc:654] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.607808 11592 raft_consensus.cc:359] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.608505 11592 raft_consensus.cc:385] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.608780 11592 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.609412 11592 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.612069 11592 ts_tablet_manager.cc:1434] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:14:45.613209 11592 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.614639 11595 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.615131 11595 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.619524 11596 tablet_bootstrap.cc:654] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.620877 11592 tablet_bootstrap.cc:654] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.631364 11596 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.631474 11592 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.631831 11596 ts_tablet_manager.cc:1403] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.002s
I20260709 13:14:45.632064 11592 ts_tablet_manager.cc:1403] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.008s
I20260709 13:14:45.632455 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.633962 11550 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:45.634742 11442 leader_election.cc:343] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.636081 11592 raft_consensus.cc:359] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.636983 11592 raft_consensus.cc:385] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.637311 11592 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.638407 11592 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.639015 11595 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.640440 11592 ts_tablet_manager.cc:1434] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:14:45.640421 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.641170 11551 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:45.641520 11592 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.642125 11549 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.644047 11596 raft_consensus.cc:359] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.644716 11596 raft_consensus.cc:385] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.644971 11596 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.645602 11596 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.646847 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:45.647586 11596 ts_tablet_manager.cc:1434] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:14:45.647620 11595 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.648104 11595 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.648521 11596 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.648442 11595 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.652107 11664 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:14:45.656457 11596 tablet_bootstrap.cc:654] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.655866 11595 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.660516 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.661069 11402 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.665598 11664 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:45.669602 11402 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.670800 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:45.671559 11651 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.672043 11651 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:45.672845 11651 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.648753 11592 tablet_bootstrap.cc:654] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.682101 11595 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.682538 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.682821 11591 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.682971 11551 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.683434 11591 ts_tablet_manager.cc:1403] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.085s	user 0.029s	sys 0.013s
I20260709 13:14:45.691934 11402 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:45.693876 11651 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:45.695040 11619 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:45.695387 11619 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:45.695644 11664 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.696201 11664 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.696256 11619 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.697868 11619 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:45.699090 11664 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.700711 11651 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.700753 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.703792 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.704018 11591 raft_consensus.cc:359] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.704871 11591 raft_consensus.cc:385] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.705147 11591 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
W20260709 13:14:45.705338 11367 leader_election.cc:343] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.705961 11591 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:45.706627 11365 leader_election.cc:343] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.707208 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.708150 11664 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:45.710052 11591 ts_tablet_manager.cc:1434] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:14:45.711265 11591 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.718858 11591 tablet_bootstrap.cc:654] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.731542 11591 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.732597 11591 ts_tablet_manager.cc:1403] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.007s
I20260709 13:14:45.735996 11659 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:45.736572 11699 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:14:45.739271 11702 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:14:45.741276 11619 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.695537 11551 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.742659 11475 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.745369 11591 raft_consensus.cc:359] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.746315 11591 raft_consensus.cc:385] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.746657 11591 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.747896 11591 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.750690 11656 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.750824 11591 ts_tablet_manager.cc:1434] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:14:45.751269 11656 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.751876 11591 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.753718 11656 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.755308 11592 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:45.755765 11592 ts_tablet_manager.cc:1403] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.114s	user 0.018s	sys 0.004s
I20260709 13:14:45.755524 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.756290 11474 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:45.758409 11592 raft_consensus.cc:359] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.759806 11591 tablet_bootstrap.cc:654] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.761371 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:45.761929 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.765913 11596 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:45.766373 11596 ts_tablet_manager.cc:1403] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.118s	user 0.044s	sys 0.052s
W20260709 13:14:45.767022 11367 leader_election.cc:343] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.768229 11592 raft_consensus.cc:385] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.768390 11656 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.768687 11619 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.769003 11656 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.769193 11619 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.769465 11656 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.769384 11596 raft_consensus.cc:359] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.770399 11596 raft_consensus.cc:385] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.770766 11596 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.771754 11619 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.730291 11297 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.771773 11596 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.774219 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.774529 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:45.776201 11442 leader_election.cc:343] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.768532 11592 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.775226 11596 ts_tablet_manager.cc:1434] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:14:45.783891 11596 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:45.783829 11659 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:45.787395 11591 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:45.787978 11591 ts_tablet_manager.cc:1403] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.007s
I20260709 13:14:45.787467 11592 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.788754 11401 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:45.790351 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:45.791666 11619 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:45.792152 11619 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.792529 11619 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.790601 11591 raft_consensus.cc:359] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.795226 11591 raft_consensus.cc:385] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:45.795562 11591 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:45.796722 11591 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.800750 11619 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.803938 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:45.804515 11401 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.802882 11596 tablet_bootstrap.cc:654] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.810285 11591 ts_tablet_manager.cc:1434] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:14:45.812876 11591 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:45.828860 11712 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:14:45.829339 11656 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.835201 11592 ts_tablet_manager.cc:1434] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.079s	user 0.016s	sys 0.013s
I20260709 13:14:45.836491 11592 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:45.843991 11606 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.845070 11606 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.838387 11619 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.859115 11401 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:45.879997 11713 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.862744 11591 tablet_bootstrap.cc:654] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.853351 11712 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.887737 11712 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.886473 11715 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.888775 11715 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.891330 11715 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.891983 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.899195 11712 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.895268 11713 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:45.900357 11592 tablet_bootstrap.cc:654] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:45.902792 11713 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.907332 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:45.907927 11475 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.908452 11606 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.911450 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:45.913359 11442 leader_election.cc:343] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.921475 11400 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:45.926646 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.927332 11548 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:45.927690 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:45.928498 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.929081 11549 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:45.929034 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:45.898655 11656 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:45.940574 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:14:45.938781 11365 leader_election.cc:343] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:45.955943 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.958068 11473 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.963315 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:45.964286 11402 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:45.956532 11619 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:45.971515 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.978866 11606 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.981292 11606 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.006349 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.009145 11606 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:45.977216 11619 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:46.010638 11619 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.017372 11516 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:14:46.018623 11516 leader_election.cc:343] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.023939 11670 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.026939 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:45.968343 11367 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:45.980993 11472 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:46.038545 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:46.027629 11477 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:46.044003 11656 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.044526 11656 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.044973 11656 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:46.047132 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 103780us
W20260709 13:14:46.048048 11367 leader_election.cc:343] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.049223 11712 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.049328 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:46.051746 11712 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.052210 11712 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.052618 11712 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.024358 11670 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.059813 11670 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.077080 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:46.061431 11606 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:14:46.112207 11656 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.114440 11656 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:46.155347 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85711.3us
W20260709 13:14:46.159291 11514 consensus_peers.cc:597] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.158049 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.129442 11712 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:46.207762 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50164.5us
I20260709 13:14:46.208135 11606 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.208930 11606 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.209373 11606 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.210137 11712 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.210737 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.211921 11477 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.215804 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.216696 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.217378 11651 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.218218 11651 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.224836 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.227656 11551 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.228987 11719 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:14:46.235658 11591 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.236524 11591 ts_tablet_manager.cc:1403] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.424s	user 0.016s	sys 0.030s
W20260709 13:14:46.238940 11365 leader_election.cc:343] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.250032 11651 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.250293 11591 raft_consensus.cc:359] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.253407 11591 raft_consensus.cc:385] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.251597 11596 tablet_bootstrap.cc:492] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.251911 11475 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.253804 11591 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.259442 11656 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:46.263674 11596 ts_tablet_manager.cc:1403] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.480s	user 0.088s	sys 0.081s
I20260709 13:14:46.271165 11591 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.276892 11591 ts_tablet_manager.cc:1434] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 13:14:46.268325 11551 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.272481 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.286940 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:46.287933 11710 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.288610 11710 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:46.289985 11592 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.290109 11716 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.290776 11592 ts_tablet_manager.cc:1403] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.455s	user 0.013s	sys 0.015s
I20260709 13:14:46.291501 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.291173 11710 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.295346 11400 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:46.299438 11650 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.302839 11596 raft_consensus.cc:359] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.303820 11596 raft_consensus.cc:385] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.304185 11596 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.305049 11596 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.293524 11716 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:46.327517 11591 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.331681 11716 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:46.332721 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65125.4us
I20260709 13:14:46.341660 11670 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.343202 11549 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:14:46.344473 11728 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:46.345217 11670 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:46.345425 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.346360 11400 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:46.346371 11606 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.349076 11477 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.349856 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.351840 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:14:46.352982 11516 leader_election.cc:343] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.361547 11619 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.383208 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.386837 11592 raft_consensus.cc:359] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.387677 11592 raft_consensus.cc:385] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.387992 11592 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.388624 11592 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.390954 11592 ts_tablet_manager.cc:1434] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.100s	user 0.007s	sys 0.000s
I20260709 13:14:46.368047 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:46.395596 11727 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.397447 11727 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.397960 11727 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.396726 11477 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:46.404160 11710 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.392930 11606 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:46.409896 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.410128 11591 tablet_bootstrap.cc:654] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.410511 11550 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:46.411263 11710 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:46.420433 11514 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:46.421293 11606 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:46.423317 11596 ts_tablet_manager.cc:1434] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.159s	user 0.008s	sys 0.000s
I20260709 13:14:46.424677 11596 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.424973 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.426199 11476 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.432430 11710 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.432756 11727 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.439245 11727 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.446519 11651 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.446893 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.448436 11548 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.453917 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.454605 11399 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.457772 11596 tablet_bootstrap.cc:654] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.457954 11548 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.459175 11592 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.463064 11399 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.471968 11297 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.472744 11476 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:46.474928 11591 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.475421 11591 ts_tablet_manager.cc:1403] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.148s	user 0.021s	sys 0.000s
I20260709 13:14:46.479552 11592 tablet_bootstrap.cc:654] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.480091 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:46.481590 11650 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.481746 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:46.482125 11650 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:46.482235 11591 raft_consensus.cc:359] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.482929 11591 raft_consensus.cc:385] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.483260 11591 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.483091 11650 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.484289 11727 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.486150 11591 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.487172 11727 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:46.490588 11591 ts_tablet_manager.cc:1434] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:14:46.490459 11727 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.491636 11591 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
W20260709 13:14:46.502799 11442 consensus_peers.cc:597] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.507764 11298 catalog_manager.cc:5697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.511973 11729 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.513738 11596 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.514384 11596 ts_tablet_manager.cc:1403] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.090s	user 0.008s	sys 0.007s
I20260709 13:14:46.519719 11298 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.527130 11298 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:46.529961 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:46.534883 11300 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.536684 11596 raft_consensus.cc:359] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.546195 11596 raft_consensus.cc:385] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.546514 11596 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.538094 11592 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.547663 11592 ts_tablet_manager.cc:1403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.095s	user 0.027s	sys 0.011s
I20260709 13:14:46.547235 11596 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.549417 11596 ts_tablet_manager.cc:1434] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:14:46.551682 11596 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.552305 11592 raft_consensus.cc:359] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.552997 11592 raft_consensus.cc:385] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.553267 11592 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.553925 11592 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.555721 11592 ts_tablet_manager.cc:1434] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:46.556540 11591 tablet_bootstrap.cc:654] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.559785 11592 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.566272 11592 tablet_bootstrap.cc:654] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.570086 11596 tablet_bootstrap.cc:654] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.573035 11297 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.577416 11592 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.577874 11592 ts_tablet_manager.cc:1403] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:14:46.580583 11592 raft_consensus.cc:359] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.581265 11592 raft_consensus.cc:385] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.581588 11596 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.581779 11591 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.581609 11592 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.582120 11596 ts_tablet_manager.cc:1403] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.002s
I20260709 13:14:46.582254 11591 ts_tablet_manager.cc:1403] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.091s	user 0.009s	sys 0.008s
I20260709 13:14:46.582826 11592 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.584728 11591 raft_consensus.cc:359] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.585361 11591 raft_consensus.cc:385] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.586391 11591 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.587181 11591 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.589406 11591 ts_tablet_manager.cc:1434] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:46.589231 11596 raft_consensus.cc:359] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.589910 11596 raft_consensus.cc:385] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.590212 11596 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.590556 11591 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.590858 11596 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.594794 11592 ts_tablet_manager.cc:1434] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:14:46.595868 11592 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.596727 11591 tablet_bootstrap.cc:654] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.597429 11596 ts_tablet_manager.cc:1434] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:14:46.598584 11596 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.601869 11592 tablet_bootstrap.cc:654] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.606096 11596 tablet_bootstrap.cc:654] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.607926 11591 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.608366 11591 ts_tablet_manager.cc:1403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:14:46.611721 11591 raft_consensus.cc:359] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.612339 11591 raft_consensus.cc:385] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.612591 11591 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.613207 11591 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.615824 11591 ts_tablet_manager.cc:1434] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:46.617183 11591 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.618203 11592 tablet_bootstrap.cc:492] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.618726 11592 ts_tablet_manager.cc:1403] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.005s
I20260709 13:14:46.621968 11592 raft_consensus.cc:359] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.622373 11596 tablet_bootstrap.cc:492] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.622835 11592 raft_consensus.cc:385] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.623061 11596 ts_tablet_manager.cc:1403] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.007s
I20260709 13:14:46.623208 11592 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.623587 11591 tablet_bootstrap.cc:654] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.624148 11592 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.626245 11592 ts_tablet_manager.cc:1434] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:46.626384 11596 raft_consensus.cc:359] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.627429 11592 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.627424 11596 raft_consensus.cc:385] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.628015 11596 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.629127 11596 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.634114 11592 tablet_bootstrap.cc:654] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.634840 11596 ts_tablet_manager.cc:1434] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:46.635869 11596 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.642138 11596 tablet_bootstrap.cc:654] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.645587 11592 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.646097 11592 ts_tablet_manager.cc:1403] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.012s
I20260709 13:14:46.647307 11591 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.647890 11591 ts_tablet_manager.cc:1403] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.005s
I20260709 13:14:46.648814 11592 raft_consensus.cc:359] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.649613 11592 raft_consensus.cc:385] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.649932 11592 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.650552 11591 raft_consensus.cc:359] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.651208 11591 raft_consensus.cc:385] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.650741 11592 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.651479 11591 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.652159 11591 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.653312 11592 ts_tablet_manager.cc:1434] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:46.654498 11592 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.657891 11591 ts_tablet_manager.cc:1434] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:14:46.659044 11591 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.662672 11592 tablet_bootstrap.cc:654] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.664973 11591 tablet_bootstrap.cc:654] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.668711 11596 tablet_bootstrap.cc:492] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.669888 11596 ts_tablet_manager.cc:1403] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.001s
I20260709 13:14:46.673502 11710 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.673310 11596 raft_consensus.cc:359] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.674204 11596 raft_consensus.cc:385] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.673992 11710 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.674563 11596 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.676328 11710 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.678117 11596 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.679474 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.680083 11551 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:46.681476 11367 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:46.678567 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:46.682498 11476 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:46.683717 11596 ts_tablet_manager.cc:1434] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:14:46.684188 11710 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.684754 11596 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.684639 11710 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.685182 11710 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.690779 11596 tablet_bootstrap.cc:654] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.692349 11710 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.695304 11650 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.695602 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.696169 11476 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.695974 11650 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.700727 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.702280 11476 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.703688 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:46.704736 11716 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.704859 11650 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:46.705243 11716 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:46.705024 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:46.706001 11476 raft_consensus.cc:2393] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:46.706104 11716 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.707188 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:46.708280 11650 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.716507 11650 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:46.718775 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.719475 11551 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.722303 11591 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.722765 11592 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.717443 11710 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.722873 11596 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.723274 11729 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.722769 11591 ts_tablet_manager.cc:1403] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.015s
I20260709 13:14:46.723663 11592 ts_tablet_manager.cc:1403] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.009s
I20260709 13:14:46.723763 11729 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.726025 11729 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.726296 11591 raft_consensus.cc:359] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.716944 11727 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.726953 11591 raft_consensus.cc:385] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.727276 11591 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.727149 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.728016 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.728036 11591 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.728763 11402 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:46.729472 11592 raft_consensus.cc:359] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.723771 11596 ts_tablet_manager.cc:1403] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.039s	user 0.027s	sys 0.011s
I20260709 13:14:46.730058 11551 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.727276 11727 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.730505 11592 raft_consensus.cc:385] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.732234 11592 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.732684 11727 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.733136 11592 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.733623 11596 raft_consensus.cc:359] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.734315 11596 raft_consensus.cc:385] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.734608 11596 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.734957 11591 ts_tablet_manager.cc:1434] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:46.735704 11592 ts_tablet_manager.cc:1434] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:14:46.735932 11591 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.735292 11596 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.737329 11596 ts_tablet_manager.cc:1434] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
W20260709 13:14:46.730975 11442 leader_election.cc:343] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.738678 11596 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.739795 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.742118 11402 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:46.742388 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.741616 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.743036 11551 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:46.743726 11729 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.744109 11729 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.744400 11729 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.748558 11727 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.749523 11727 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.752712 11591 tablet_bootstrap.cc:654] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.753187 11592 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:46.751706 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:46.758626 11727 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.758890 11651 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.759402 11651 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.759794 11596 tablet_bootstrap.cc:654] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.760043 11651 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.759269 11716 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.763487 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.764871 11729 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.767362 11729 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.762570 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.768293 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.769148 11399 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.769145 11402 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
W20260709 13:14:46.770076 11442 leader_election.cc:343] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.771189 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.771879 11476 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.773038 11721 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.773598 11721 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.774679 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.768397 11298 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.775970 11729 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.780377 11729 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.780825 11729 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.785606 11721 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:46.789901 11716 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.791487 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:46.795435 11516 leader_election.cc:343] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.802914 11716 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.803997 11399 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.773694 11651 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.814041 11592 tablet_bootstrap.cc:654] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.806787 11651 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:46.783983 11442 leader_election.cc:343] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.818471 11729 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.818950 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.822273 11402 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.822660 11729 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.823272 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.823844 11550 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.827986 11591 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:46.828423 11591 ts_tablet_manager.cc:1403] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.093s	user 0.017s	sys 0.008s
I20260709 13:14:46.846072 11402 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.847146 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.850425 11727 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.850548 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:46.850935 11727 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:46.851644 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:46.851833 11727 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.854077 11651 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.854532 11651 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:46.854660 11550 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.855377 11651 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:46.857491 11514 leader_election.cc:343] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.858029 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:46.857874 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:46.861696 11596 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.859550 11721 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:46.863319 11596 ts_tablet_manager.cc:1403] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.125s	user 0.029s	sys 0.015s
I20260709 13:14:46.867365 11596 raft_consensus.cc:359] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.868062 11596 raft_consensus.cc:385] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.868574 11596 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.869338 11596 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.871354 11596 ts_tablet_manager.cc:1434] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:14:46.861732 11399 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.872665 11596 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.871367 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.878099 11729 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.879879 11591 raft_consensus.cc:359] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.880589 11591 raft_consensus.cc:385] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.880810 11591 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.881215 11729 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.884814 11596 tablet_bootstrap.cc:654] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.885758 11591 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.891571 11591 ts_tablet_manager.cc:1434] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:14:46.892663 11591 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:46.884799 11729 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.896279 11710 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.901275 11710 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:46.904949 11442 leader_election.cc:343] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.908905 11715 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:46.909327 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:46.909495 11710 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.909925 11476 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:46.916090 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:46.920017 11367 leader_election.cc:343] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.916177 11399 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:46.921545 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.925251 11710 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.925699 11710 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.925472 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:46.926223 11710 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:46.929282 11516 consensus_peers.cc:597] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.935812 11710 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.938583 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:46.939198 11476 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.943667 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:46.945026 11710 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:46.945359 11367 leader_election.cc:343] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:46.949107 11596 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:46.951577 11596 ts_tablet_manager.cc:1403] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.018s
I20260709 13:14:46.951716 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.951629 11591 tablet_bootstrap.cc:654] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.955597 11596 raft_consensus.cc:359] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.956353 11596 raft_consensus.cc:385] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:46.956660 11596 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:46.957427 11596 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.957610 11298 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.959486 11596 ts_tablet_manager.cc:1434] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:14:46.959128 11298 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.960700 11596 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:46.961501 11710 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:46.965236 11476 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:46.985092 11716 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.985608 11716 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:46.991817 11592 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:46.992491 11592 ts_tablet_manager.cc:1403] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.240s	user 0.016s	sys 0.034s
I20260709 13:14:46.992673 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:46.993297 11551 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:46.994441 11367 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:46.995218 11716 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:46.995738 11716 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:46.995816 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:46.996152 11716 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.996611 11727 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:46.996800 11716 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:46.997098 11727 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:46.998005 11727 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.002431 11596 tablet_bootstrap.cc:654] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:46.995043 11592 raft_consensus.cc:359] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.004594 11592 raft_consensus.cc:385] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.004860 11592 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.005144 11729 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.006842 11592 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.009609 11592 ts_tablet_manager.cc:1434] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.003s
I20260709 13:14:47.011379 11592 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.011488 11710 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.012038 11710 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.014325 11596 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.014876 11596 ts_tablet_manager.cc:1403] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.001s
I20260709 13:14:47.017447 11592 tablet_bootstrap.cc:654] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.017445 11596 raft_consensus.cc:359] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.018373 11596 raft_consensus.cc:385] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.018728 11596 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.019470 11596 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.021643 11596 ts_tablet_manager.cc:1434] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:14:47.022851 11596 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.029340 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.029978 11596 tablet_bootstrap.cc:654] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.030660 11715 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.034228 11716 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.037093 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.038434 11715 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:47.039060 11715 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:47.039984 11715 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.045490 11477 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.046357 11596 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.046802 11596 ts_tablet_manager.cc:1403] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 13:14:47.048805 11592 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:47.049232 11592 ts_tablet_manager.cc:1403] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.001s
I20260709 13:14:47.049271 11596 raft_consensus.cc:359] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.050006 11596 raft_consensus.cc:385] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.050246 11596 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.050875 11596 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.051753 11592 raft_consensus.cc:359] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.052367 11592 raft_consensus.cc:385] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.052600 11592 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.053047 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.053313 11592 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.053805 11710 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.053711 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.054495 11475 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:14:47.060004 11596 ts_tablet_manager.cc:1434] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:14:47.060968 11442 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.061417 11596 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.065040 11710 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.065600 11710 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.072027 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.072825 11729 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:47.075973 11365 leader_election.cc:343] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.079946 11592 ts_tablet_manager.cc:1434] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.008s
I20260709 13:14:47.087536 11592 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.083544 11299 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:47.078090 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.089090 11721 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.092103 11721 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.094533 11721 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.096843 11606 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.098843 11596 tablet_bootstrap.cc:654] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.099220 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.099457 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.100208 11716 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:47.103137 11365 leader_election.cc:343] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.103324 11713 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:47.103683 11713 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.104005 11713 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:47.091641 11710 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.104454 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.092418 11778 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.107388 11778 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.109746 11591 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.110239 11591 ts_tablet_manager.cc:1403] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.218s	user 0.017s	sys 0.011s
I20260709 13:14:47.111280 11721 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.111871 11721 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.113098 11606 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:14:47.113734 11365 leader_election.cc:343] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.113651 11606 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.114773 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.118697 11670 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.140146 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.156678 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.173830 11550 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:47.180728 11670 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.181419 11670 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.210979 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.218238 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.220197 11591 raft_consensus.cc:359] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.221490 11606 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.223059 11670 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.223481 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.224082 11778 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.224138 11399 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
W20260709 13:14:47.225281 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 110626us
I20260709 13:14:47.225565 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.225540 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:47.227311 11715 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.227706 11715 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.228051 11715 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.228546 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.229172 11547 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:47.229179 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.229924 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.230552 11477 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:14:47.232551 11367 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:47.233467 11716 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.234018 11716 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.234452 11716 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.234573 11721 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.235201 11721 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.235531 11721 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.235867 11721 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.239550 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:47.241439 11516 leader_election.cc:343] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.243788 11591 raft_consensus.cc:385] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:47.244309 11365 leader_election.cc:343] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.219105 11550 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.246132 11591 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.257927 11592 tablet_bootstrap.cc:654] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.249316 11298 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:47.263792 11514 leader_election.cc:343] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.264782 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:47.263840 11551 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.265561 11670 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:47.265940 11670 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:14:47.266270 11670 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:47.261526 11591 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.272738 11591 ts_tablet_manager.cc:1434] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.162s	user 0.006s	sys 0.003s
I20260709 13:14:47.273790 11591 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.274132 11715 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.276577 11777 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.281067 11713 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.285362 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.287894 11596 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.288352 11402 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:14:47.281454 11783 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:47.290134 11596 ts_tablet_manager.cc:1403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.229s	user 0.027s	sys 0.023s
I20260709 13:14:47.288956 11721 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:47.293260 11365 leader_election.cc:343] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.276826 11715 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.295445 11516 leader_election.cc:400] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:14:47.297199 11670 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.297626 11596 raft_consensus.cc:359] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.299669 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.310776 11548 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.312343 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.312948 11473 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:47.300856 11713 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.302488 11550 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:47.302441 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.303956 11716 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.323896 11596 raft_consensus.cc:385] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.324333 11596 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.321022 11549 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:47.326535 11778 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.330893 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:47.331656 11402 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:14:47.332777 11367 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.333317 11716 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.333904 11716 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:47.334537 11716 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:47.335251 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.333124 11516 leader_election.cc:400] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:14:47.337419 11592 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
W20260709 13:14:47.337528 11365 leader_election.cc:343] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.337929 11592 ts_tablet_manager.cc:1403] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.257s	user 0.026s	sys 0.006s
I20260709 13:14:47.325186 11596 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.338429 11516 leader_election.cc:403] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:14:47.342144 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.348652 11596 ts_tablet_manager.cc:1434] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.000s
I20260709 13:14:47.348666 11550 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:47.354975 11713 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.355511 11713 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.357687 11367 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.359661 11783 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.360117 11783 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.359688 11778 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:14:47.359688 11442 consensus_peers.cc:597] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:47.368777 11596 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.372113 11592 raft_consensus.cc:359] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.373705 11592 raft_consensus.cc:385] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.373997 11592 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.374706 11721 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.374949 11592 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.375510 11783 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.368852 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.377679 11592 ts_tablet_manager.cc:1434] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.008s
I20260709 13:14:47.378700 11592 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.371120 11716 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.381028 11591 tablet_bootstrap.cc:654] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.378633 11549 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:14:47.385272 11650 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:14:47.399677 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:47.402364 11365 leader_election.cc:343] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.392940 11670 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:14:47.421630 11514 leader_election.cc:343] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.422150 11514 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.423871 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.426553 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:47.426513 11619 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:47.428107 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.417001 11721 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.418946 11713 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.442624 11400 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:47.446979 11651 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.447362 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.436926 11670 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.454202 11596 tablet_bootstrap.cc:654] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.418604 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:14:47.458243 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53270us
I20260709 13:14:47.458830 11778 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:47.453423 11721 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.462211 11778 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:47.462760 11546 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.463286 11721 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.463616 11619 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:14:47.478720 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.486032 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:47.472172 11473 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:14:47.495160 11514 leader_election.cc:343] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.504241 11778 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.495635 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.509577 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.512567 11402 raft_consensus.cc:2393] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:14:47.516866 11788 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:14:47.518499 11591 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.518890 11591 ts_tablet_manager.cc:1403] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.245s	user 0.017s	sys 0.019s
I20260709 13:14:47.513790 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.526901 11549 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:47.528957 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.533821 11619 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.537777 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.541071 11715 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:47.541587 11715 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:47.542409 11715 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:47.543877 11367 leader_election.cc:343] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.528401 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.547816 11367 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.560513 11596 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.560972 11596 ts_tablet_manager.cc:1403] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.204s	user 0.019s	sys 0.011s
I20260709 13:14:47.561296 11592 tablet_bootstrap.cc:654] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.566255 11778 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.566659 11298 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:47.561468 11650 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.577667 11591 raft_consensus.cc:359] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.578361 11591 raft_consensus.cc:385] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.578608 11591 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
W20260709 13:14:47.589763 11367 consensus_peers.cc:597] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:47.568084 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44625.1us
I20260709 13:14:47.608238 11516 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.615397 11650 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.616108 11619 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.621085 11596 raft_consensus.cc:359] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.621830 11596 raft_consensus.cc:385] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.621938 11650 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:47.622121 11801 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:14:47.622108 11596 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.624264 11596 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.622700 11650 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.632364 11619 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.634667 11650 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:47.637521 11777 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.631197 11591 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.640156 11591 ts_tablet_manager.cc:1434] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.121s	user 0.007s	sys 0.000s
I20260709 13:14:47.641090 11591 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.643838 11596 ts_tablet_manager.cc:1434] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.000s
I20260709 13:14:47.644757 11596 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.644542 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.649117 11592 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:47.649564 11592 ts_tablet_manager.cc:1403] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.271s	user 0.011s	sys 0.007s
I20260709 13:14:47.650028 11783 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.643646 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.653406 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.654211 11650 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.655619 11596 tablet_bootstrap.cc:654] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.661124 11592 raft_consensus.cc:359] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.662003 11592 raft_consensus.cc:385] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.662289 11592 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.663089 11592 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.665329 11592 ts_tablet_manager.cc:1434] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
W20260709 13:14:47.665308 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:47.670333 11591 tablet_bootstrap.cc:654] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.671281 11592 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.668963 11297 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:47.672325 11297 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:47.696218 11591 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.696755 11591 ts_tablet_manager.cc:1403] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.012s
I20260709 13:14:47.696781 11596 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.697338 11596 ts_tablet_manager.cc:1403] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.005s
I20260709 13:14:47.698881 11592 tablet_bootstrap.cc:654] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.699635 11591 raft_consensus.cc:359] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.700300 11591 raft_consensus.cc:385] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.700542 11591 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.701280 11591 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.703338 11591 ts_tablet_manager.cc:1434] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:14:47.704749 11591 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.705391 11778 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.705336 11596 raft_consensus.cc:359] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.706085 11596 raft_consensus.cc:385] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.705885 11778 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.706349 11596 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.706959 11596 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.707870 11778 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.709131 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.709656 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.711159 11591 tablet_bootstrap.cc:654] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.711270 11367 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.712868 11778 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.715276 11596 ts_tablet_manager.cc:1434] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:14:47.717852 11596 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.725780 11596 tablet_bootstrap.cc:654] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.734601 11592 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:47.736315 11591 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.736848 11591 ts_tablet_manager.cc:1403] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.022s
I20260709 13:14:47.739557 11591 raft_consensus.cc:359] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.740257 11591 raft_consensus.cc:385] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.740562 11591 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.741235 11591 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.735128 11592 ts_tablet_manager.cc:1403] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.001s
I20260709 13:14:47.743935 11591 ts_tablet_manager.cc:1434] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:47.744936 11591 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.746222 11592 raft_consensus.cc:359] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.747182 11592 raft_consensus.cc:385] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.747524 11592 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.748396 11592 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.751849 11591 tablet_bootstrap.cc:654] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.751861 11596 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.752565 11596 ts_tablet_manager.cc:1403] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.018s
I20260709 13:14:47.752058 11592 ts_tablet_manager.cc:1434] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:14:47.754587 11596 raft_consensus.cc:359] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.755314 11596 raft_consensus.cc:385] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.755586 11596 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.756167 11592 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.756328 11596 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.761987 11592 tablet_bootstrap.cc:654] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.763056 11591 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.763507 11591 ts_tablet_manager.cc:1403] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:14:47.766109 11591 raft_consensus.cc:359] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.766752 11591 raft_consensus.cc:385] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.767117 11591 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.767700 11591 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.769376 11596 ts_tablet_manager.cc:1434] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.001s
I20260709 13:14:47.769748 11591 ts_tablet_manager.cc:1434] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:47.770715 11596 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.770711 11591 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.776932 11549 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.789476 11619 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.790011 11619 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.791925 11619 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.794055 11778 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.803210 11591 tablet_bootstrap.cc:654] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.803534 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.805656 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:47.807399 11442 leader_election.cc:343] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.813529 11778 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.814288 11778 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.817945 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.818717 11596 tablet_bootstrap.cc:654] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.819129 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:47.819422 11365 leader_election.cc:343] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:47.820848 11367 leader_election.cc:343] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.821280 11367 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.821950 11591 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.822070 11715 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.822477 11591 ts_tablet_manager.cc:1403] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.003s
I20260709 13:14:47.824436 11592 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:47.825047 11592 ts_tablet_manager.cc:1403] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.020s
I20260709 13:14:47.825896 11591 raft_consensus.cc:359] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.826812 11591 raft_consensus.cc:385] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.827176 11591 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.827258 11632 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.828678 11592 raft_consensus.cc:359] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.829492 11592 raft_consensus.cc:385] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.829324 11632 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.829205 11591 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.829876 11592 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.830889 11592 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.831452 11632 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.832172 11591 ts_tablet_manager.cc:1434] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:14:47.832580 11592 ts_tablet_manager.cc:1434] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:14:47.833535 11592 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.837105 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:47.840467 11442 leader_election.cc:343] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.841836 11778 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.849620 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.850294 11546 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
W20260709 13:14:47.852909 11442 leader_election.cc:343] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.853363 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:47.854113 11729 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:47.855855 11591 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.857107 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:47.860955 11729 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.861428 11729 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.862442 11729 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.866142 11591 tablet_bootstrap.cc:654] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.887898 11729 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.886897 11592 tablet_bootstrap.cc:654] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.890631 11729 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.900478 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.901139 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:47.901386 11549 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.902863 11778 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.903466 11778 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:47.904439 11442 leader_election.cc:343] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.904649 11596 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:47.914321 11596 ts_tablet_manager.cc:1403] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.144s	user 0.039s	sys 0.040s
I20260709 13:14:47.905862 11778 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.907398 11592 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:47.907276 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.908389 11715 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.916361 11546 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:47.916775 11715 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.917356 11596 raft_consensus.cc:359] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.917619 11367 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:47.918030 11596 raft_consensus.cc:385] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.918356 11596 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.918500 11777 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.918789 11715 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.918926 11777 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.919363 11777 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.919018 11596 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.910243 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.919759 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:47.919499 11592 ts_tablet_manager.cc:1403] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.006s
W20260709 13:14:47.920611 11365 leader_election.cc:343] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.921105 11591 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:47.921751 11591 ts_tablet_manager.cc:1403] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.069s	user 0.020s	sys 0.010s
W20260709 13:14:47.922139 11365 leader_election.cc:343] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.915225 11716 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:47.923226 11729 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.923792 11729 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.924232 11591 raft_consensus.cc:359] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.924883 11591 raft_consensus.cc:385] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.925120 11596 ts_tablet_manager.cc:1434] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:47.925982 11729 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.926146 11596 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:47.923063 11592 raft_consensus.cc:359] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.926537 11592 raft_consensus.cc:385] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:47.926811 11592 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.927472 11592 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.928872 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:47.929510 11592 ts_tablet_manager.cc:1434] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:14:47.929474 11402 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:47.929956 11549 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:47.930876 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.943133 11546 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:47.932080 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:47.932727 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:47.944015 11545 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:47.944626 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:47.945348 11619 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.945680 11619 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.945990 11619 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.946106 11367 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:47.947199 11716 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:47.947556 11716 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.947871 11716 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.949162 11729 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:47.949666 11729 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:47.950495 11729 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.954170 11619 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.964829 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:47.965423 11402 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.937099 11592 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:47.940579 11777 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.970767 11777 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.925148 11591 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:47.971899 11402 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:47.954372 11716 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.973915 11619 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.973802 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.974792 11592 tablet_bootstrap.cc:654] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.974846 11547 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.975077 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.936124 11596 tablet_bootstrap.cc:654] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:47.973840 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:47.977839 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:47.978356 11783 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:14:47.977252 11365 leader_election.cc:343] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.978825 11783 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:14:47.979369 11365 leader_election.cc:343] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:47.971647 11591 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.979687 11783 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:47.981599 11591 ts_tablet_manager.cc:1434] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 13:14:47.982627 11591 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:47.974095 11716 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:47.995608 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:47.996233 11546 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:47.997951 11547 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:47.999389 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:48.003726 11592 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.004148 11592 ts_tablet_manager.cc:1403] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.001s
I20260709 13:14:48.004667 11549 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.006382 11297 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.007326 11592 raft_consensus.cc:359] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.008019 11592 raft_consensus.cc:385] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.008283 11592 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.008819 11592 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.011763 11592 ts_tablet_manager.cc:1434] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:48.012998 11592 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.014442 11549 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:48.020243 11367 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:48.021093 11546 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:48.021660 11716 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.022876 11591 tablet_bootstrap.cc:654] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.023526 11777 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.023825 11477 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.024667 11716 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:48.029922 11299 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.027570 11716 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.033394 11596 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.033946 11596 ts_tablet_manager.cc:1403] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.108s	user 0.015s	sys 0.003s
I20260709 13:14:48.028385 11367 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:48.036921 11777 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.037441 11777 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:48.038749 11777 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.041544 11592 tablet_bootstrap.cc:654] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.046685 11596 raft_consensus.cc:359] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.047832 11596 raft_consensus.cc:385] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.048269 11596 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.052757 11591 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.053468 11591 ts_tablet_manager.cc:1403] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.000s
I20260709 13:14:48.049600 11596 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.063599 11596 ts_tablet_manager.cc:1434] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:14:48.064628 11596 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.069190 11710 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.070825 11547 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:48.072674 11710 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.075763 11650 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.076190 11596 tablet_bootstrap.cc:654] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.076251 11650 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.076334 11591 raft_consensus.cc:359] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.077306 11591 raft_consensus.cc:385] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.077580 11591 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.078260 11591 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.079741 11473 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:48.079823 11591 ts_tablet_manager.cc:1434] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:14:48.081033 11591 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.083823 11716 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.086174 11650 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:48.088534 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.088593 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:48.091614 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:48.093235 11650 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:48.101877 11591 tablet_bootstrap.cc:654] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.111411 11299 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.114070 11299 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.134716 11596 tablet_bootstrap.cc:492] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.135335 11596 ts_tablet_manager.cc:1403] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.071s	user 0.006s	sys 0.016s
I20260709 13:14:48.137588 11729 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.138144 11729 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.150133 11729 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.150699 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.151332 11402 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:48.153602 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:48.154264 11547 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.154011 11596 raft_consensus.cc:359] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.154870 11596 raft_consensus.cc:385] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.155161 11596 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.155783 11596 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.154472 11729 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.158799 11729 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.159224 11729 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.161942 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.162580 11546 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:48.165946 11596 ts_tablet_manager.cc:1434] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 13:14:48.168612 11592 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.169065 11592 ts_tablet_manager.cc:1403] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.156s	user 0.035s	sys 0.029s
I20260709 13:14:48.170462 11727 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.170912 11727 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.171743 11592 raft_consensus.cc:359] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.172372 11592 raft_consensus.cc:385] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.172623 11592 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.174255 11727 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.175308 11591 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.175355 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.175861 11591 ts_tablet_manager.cc:1403] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.003s
I20260709 13:14:48.176805 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.178109 11729 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.178435 11591 raft_consensus.cc:359] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.179073 11591 raft_consensus.cc:385] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.179351 11591 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.180151 11591 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.182108 11591 ts_tablet_manager.cc:1434] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:48.183012 11591 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.184816 11592 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.186237 11402 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:48.187067 11400 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.186852 11592 ts_tablet_manager.cc:1434] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:14:48.188478 11596 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
W20260709 13:14:48.181716 11442 leader_election.cc:343] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.191377 11592 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.190806 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.192849 11727 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.190256 11713 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.193215 11727 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.196640 11783 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.203675 11783 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.205574 11729 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.205832 11783 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.199754 11713 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.213160 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.214003 11399 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:48.214471 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:48.215091 11549 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.217286 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.217891 11551 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:48.219672 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:48.220266 11477 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:48.221643 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:48.214672 11713 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.222640 11716 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.223023 11716 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.223361 11716 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.224979 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.226269 11545 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:48.227830 11591 tablet_bootstrap.cc:654] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.228175 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:48.228679 11402 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.231076 11716 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.206739 11727 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.235631 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:48.237228 11477 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.238418 11716 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.240908 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.245229 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:48.248309 11549 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.252671 11729 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.255158 11477 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:48.256259 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:48.257047 11777 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.257504 11777 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:48.258270 11777 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.259501 11402 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.259696 11729 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:48.260413 11547 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.263584 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:48.264328 11632 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.264796 11632 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:48.266103 11592 tablet_bootstrap.cc:654] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.267261 11632 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.271025 11547 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:48.274425 11591 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.274725 11591 ts_tablet_manager.cc:1403] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.000s
I20260709 13:14:48.281699 11591 raft_consensus.cc:359] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.282572 11591 raft_consensus.cc:385] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.282927 11591 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.283790 11591 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.285619 11297 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.286695 11596 tablet_bootstrap.cc:654] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.287106 11591 ts_tablet_manager.cc:1434] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:14:48.290927 11727 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.292553 11299 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.294592 11727 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.296540 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:48.297046 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:14:48.298687 11442 leader_election.cc:343] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.303146 11591 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.304387 11402 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.308610 11591 tablet_bootstrap.cc:654] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.311395 11402 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.312366 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.312697 11592 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.312983 11783 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.313143 11592 ts_tablet_manager.cc:1403] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.122s	user 0.004s	sys 0.006s
I20260709 13:14:48.313467 11783 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:48.314342 11783 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.315642 11592 raft_consensus.cc:359] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.316277 11592 raft_consensus.cc:385] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.316547 11592 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.317101 11592 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.317935 11596 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.318361 11596 ts_tablet_manager.cc:1403] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.130s	user 0.007s	sys 0.004s
I20260709 13:14:48.319284 11592 ts_tablet_manager.cc:1434] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:14:48.320199 11592 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.320670 11596 raft_consensus.cc:359] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.321360 11596 raft_consensus.cc:385] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.322856 11596 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.323657 11596 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.325493 11596 ts_tablet_manager.cc:1434] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:48.327791 11591 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.328248 11591 ts_tablet_manager.cc:1403] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.003s
I20260709 13:14:48.328511 11592 tablet_bootstrap.cc:654] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.330622 11591 raft_consensus.cc:359] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.331249 11591 raft_consensus.cc:385] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.331524 11591 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.332214 11591 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.334272 11591 ts_tablet_manager.cc:1434] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:48.334331 11592 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.335129 11592 ts_tablet_manager.cc:1403] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 13:14:48.335937 11596 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.340502 11592 raft_consensus.cc:359] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.341197 11592 raft_consensus.cc:385] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.341503 11592 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.344182 11596 tablet_bootstrap.cc:654] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.345413 11299 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.342212 11592 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.348466 11592 ts_tablet_manager.cc:1434] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:14:48.335140 11591 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.353235 11592 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.359172 11592 tablet_bootstrap.cc:654] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.364203 11596 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.364665 11596 ts_tablet_manager.cc:1403] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.004s
I20260709 13:14:48.366387 11591 tablet_bootstrap.cc:654] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.367368 11596 raft_consensus.cc:359] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.369597 11596 raft_consensus.cc:385] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.369868 11596 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.370563 11596 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.373780 11596 ts_tablet_manager.cc:1434] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:14:48.374711 11596 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.383733 11591 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.384269 11591 ts_tablet_manager.cc:1403] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.002s
I20260709 13:14:48.384547 11596 tablet_bootstrap.cc:654] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.385643 11592 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.386862 11591 raft_consensus.cc:359] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.389011 11591 raft_consensus.cc:385] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.389031 11592 ts_tablet_manager.cc:1403] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 13:14:48.389441 11591 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.390134 11591 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.392459 11591 ts_tablet_manager.cc:1434] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:14:48.392336 11592 raft_consensus.cc:359] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.393045 11592 raft_consensus.cc:385] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.393339 11592 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.393568 11591 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.394191 11592 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.396799 11596 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.397267 11596 ts_tablet_manager.cc:1403] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:14:48.400152 11591 tablet_bootstrap.cc:654] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.400919 11596 raft_consensus.cc:359] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.401576 11596 raft_consensus.cc:385] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.401821 11596 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.402930 11783 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:48.416031 11713 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.416778 11713 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.421382 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:48.423532 11365 leader_election.cc:343] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.447480 11596 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.449887 11596 ts_tablet_manager.cc:1434] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.052s	user 0.015s	sys 0.008s
I20260709 13:14:48.451225 11596 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.451663 11783 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.452346 11783 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.456015 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.457962 11596 tablet_bootstrap.cc:654] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.459800 11783 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.461479 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.462263 11547 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:48.463274 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.464291 11783 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.471390 11401 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.475564 11727 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:48.477329 11592 ts_tablet_manager.cc:1434] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.088s	user 0.005s	sys 0.000s
I20260709 13:14:48.481030 11713 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.484328 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.506114 11783 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:48.503933 11367 leader_election.cc:343] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.499394 11651 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.507488 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.507556 11651 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.509312 11777 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:48.509591 11651 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.511531 11592 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.511745 11651 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.513376 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.513898 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.514213 11399 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
W20260709 13:14:48.515745 11442 leader_election.cc:343] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.516430 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.515848 11651 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:48.517269 11729 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.517778 11729 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.518195 11729 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.547307 11777 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:48.550201 11591 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.550274 11592 tablet_bootstrap.cc:654] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.551108 11591 ts_tablet_manager.cc:1403] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.158s	user 0.034s	sys 0.087s
I20260709 13:14:48.556959 11596 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.557375 11596 ts_tablet_manager.cc:1403] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.106s	user 0.010s	sys 0.016s
I20260709 13:14:48.544639 11729 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.568140 11716 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.568214 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:48.568738 11716 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.568830 11402 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.571352 11651 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.572794 11716 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.572670 11651 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.574630 11651 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.575294 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:48.580313 11367 leader_election.cc:343] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.585235 11473 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.586822 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.582049 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:48.589033 11365 leader_election.cc:343] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.589450 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.603461 11596 raft_consensus.cc:359] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.610898 11651 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.627928 11596 raft_consensus.cc:385] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.628245 11596 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.628206 11651 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.629004 11596 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.601321 11777 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:48.610630 11591 raft_consensus.cc:359] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.631533 11591 raft_consensus.cc:385] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.631397 11777 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:48.631882 11442 leader_election.cc:343] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.631939 11591 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.633730 11591 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.634733 11592 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.636107 11592 ts_tablet_manager.cc:1403] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.138s	user 0.023s	sys 0.002s
W20260709 13:14:48.630076 11577 tablet.cc:2406] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:14:48.630589 11729 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.639256 11591 ts_tablet_manager.cc:1434] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.001s
I20260709 13:14:48.640247 11591 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.640383 11592 raft_consensus.cc:359] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.641047 11592 raft_consensus.cc:385] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.641295 11592 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.642239 11592 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.644233 11592 ts_tablet_manager.cc:1434] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:14:48.646564 11402 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.654558 11592 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.655848 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:48.660759 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.661497 11547 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:48.663106 11727 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.663636 11727 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:48.664456 11727 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.670040 11777 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.683086 11592 tablet_bootstrap.cc:654] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.686488 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:48.690670 11596 ts_tablet_manager.cc:1434] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.133s	user 0.005s	sys 0.008s
I20260709 13:14:48.691601 11632 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.694439 11727 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.696583 11727 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.697019 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.698632 11727 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.696074 11632 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.705605 11596 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.699303 11632 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.719156 11650 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.721621 11650 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.723315 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:48.724357 11650 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:48.725194 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:48.746073 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:48.747546 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:48.748454 11399 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
W20260709 13:14:48.754915 11442 leader_election.cc:343] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.750633 11651 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:48.717972 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 64077.1us
I20260709 13:14:48.765074 11591 tablet_bootstrap.cc:654] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:48.765296 11777 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.760912 11516 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:48.769227 11592 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.769800 11592 ts_tablet_manager.cc:1403] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.115s	user 0.020s	sys 0.004s
I20260709 13:14:48.770165 11861 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.770653 11861 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.778540 11858 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.779102 11858 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.772765 11861 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.782639 11592 raft_consensus.cc:359] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.783758 11592 raft_consensus.cc:385] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.784235 11592 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.785250 11592 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.787312 11858 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:48.787644 11592 ts_tablet_manager.cc:1434] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:14:48.790141 11592 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.786761 11632 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.805349 11632 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.806691 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:48.823796 11442 leader_election.cc:343] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.820526 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:48.828481 11545 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:48.840612 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:48.821547 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:48.849145 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:48.850261 11596 tablet_bootstrap.cc:654] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:48.851560 11514 leader_election.cc:343] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:48.852253 11514 leader_election.cc:343] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.844851 11861 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.853773 11551 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.856359 11442 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:48.858366 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:48.857825 11651 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:48.859553 11863 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:48.860249 11863 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.860586 11863 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.857095 11298 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.861356 11727 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:48.870361 11861 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:48.871457 11592 tablet_bootstrap.cc:654] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:48.871618 11442 consensus_peers.cc:597] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.873617 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:48.875451 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:48.876959 11514 leader_election.cc:343] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.880462 11863 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.885849 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:14:48.888018 11516 leader_election.cc:343] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:48.889592 11442 leader_election.cc:343] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.888528 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:48.904229 11863 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:48.908321 11716 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:48.911099 11592 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:48.911574 11592 ts_tablet_manager.cc:1403] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.122s	user 0.007s	sys 0.013s
I20260709 13:14:48.911144 11650 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:48.911850 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:14:48.907706 11867 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:48.913821 11546 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.913663 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:48.917961 11397 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:48.918426 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:48.921438 11651 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:48.922305 11442 leader_election.cc:343] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:48.923635 11651 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.928032 11546 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.932538 11869 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:48.935141 11591 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:48.935858 11591 ts_tablet_manager.cc:1403] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.296s	user 0.012s	sys 0.009s
I20260709 13:14:48.946491 11397 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:48.955444 11596 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:48.956884 11551 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:48.961982 11596 ts_tablet_manager.cc:1403] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.268s	user 0.016s	sys 0.007s
I20260709 13:14:48.962378 11592 raft_consensus.cc:359] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.964217 11592 raft_consensus.cc:385] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.964495 11592 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.965113 11592 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.965358 11516 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:48.966521 11861 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.967031 11861 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:48.967811 11861 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.969573 11596 raft_consensus.cc:359] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.970465 11596 raft_consensus.cc:385] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.970801 11596 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.971349 11592 ts_tablet_manager.cc:1434] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.002s
I20260709 13:14:48.973183 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:48.974529 11727 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.975260 11727 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:48.971633 11596 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.978051 11592 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:48.978307 11727 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.980093 11591 raft_consensus.cc:359] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.983038 11591 raft_consensus.cc:385] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:48.983310 11591 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:48.984540 11596 ts_tablet_manager.cc:1434] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 13:14:48.984308 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:48.985538 11596 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:48.983942 11591 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:48.989795 11591 ts_tablet_manager.cc:1434] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:14:48.987030 11863 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:48.993670 11863 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:48.993965 11591 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:48.995376 11863 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.014684 11591 tablet_bootstrap.cc:654] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.014988 11592 tablet_bootstrap.cc:654] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.024499 11596 tablet_bootstrap.cc:654] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.043653 11592 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:49.044183 11592 ts_tablet_manager.cc:1403] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.004s
I20260709 13:14:49.047348 11299 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.051800 11299 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.049252 11863 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.054405 11863 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.056711 11863 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.059700 11592 raft_consensus.cc:359] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.061064 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.067472 11592 raft_consensus.cc:385] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.061539 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.068331 11592 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
W20260709 13:14:49.068710 11442 leader_election.cc:343] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.068845 11399 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:49.070884 11592 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.072468 11592 ts_tablet_manager.cc:1434] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:14:49.073390 11592 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:49.075459 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.076102 11866 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.076457 11866 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.076800 11866 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.078799 11596 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.079278 11596 ts_tablet_manager.cc:1403] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.007s
I20260709 13:14:49.082060 11591 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.081856 11596 raft_consensus.cc:359] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.082497 11591 ts_tablet_manager.cc:1403] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.003s
I20260709 13:14:49.082489 11596 raft_consensus.cc:385] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.082965 11596 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.083762 11596 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.083932 11866 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.085203 11591 raft_consensus.cc:359] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.085848 11591 raft_consensus.cc:385] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.086022 11596 ts_tablet_manager.cc:1434] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.005s
I20260709 13:14:49.086113 11591 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.086799 11591 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.088023 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:49.088356 11596 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:49.088603 11399 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.088218 11298 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.091231 11592 tablet_bootstrap.cc:654] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.088927 11591 ts_tablet_manager.cc:1434] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:49.096330 11866 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.097440 11596 tablet_bootstrap.cc:654] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.103425 11591 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.106518 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:49.108858 11399 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
W20260709 13:14:49.109658 11442 leader_election.cc:343] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.111135 11591 tablet_bootstrap.cc:654] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.114221 11716 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.114753 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.114696 11716 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.117197 11716 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.119392 11727 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.119882 11727 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:49.120344 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.120661 11727 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.122745 11783 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:49.122849 11365 leader_election.cc:343] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.123227 11783 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.123967 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.125154 11783 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.126878 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.127014 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:49.128178 11367 leader_election.cc:343] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.128747 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.129860 11716 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:49.128710 11442 leader_election.cc:343] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:49.131171 11442 leader_election.cc:343] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.131735 11442 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.132690 11866 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.134382 11866 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:49.138882 11592 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:49.139444 11592 ts_tablet_manager.cc:1403] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.006s
I20260709 13:14:49.142586 11592 raft_consensus.cc:359] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.143530 11592 raft_consensus.cc:385] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.143869 11592 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.144630 11592 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.147284 11592 ts_tablet_manager.cc:1434] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:49.149067 11592 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:49.149150 11298 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:49.160738 11442 consensus_peers.cc:597] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:49.166538 11591 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.167018 11591 ts_tablet_manager.cc:1403] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.064s	user 0.001s	sys 0.025s
I20260709 13:14:49.168746 11592 tablet_bootstrap.cc:654] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.169533 11591 raft_consensus.cc:359] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.170220 11591 raft_consensus.cc:385] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.170511 11591 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.171618 11591 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.174923 11591 ts_tablet_manager.cc:1434] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:14:49.175913 11591 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.177809 11596 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.178270 11596 ts_tablet_manager.cc:1403] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.091s	user 0.019s	sys 0.011s
I20260709 13:14:49.181416 11596 raft_consensus.cc:359] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.182040 11596 raft_consensus.cc:385] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.182068 11716 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.182366 11596 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.182601 11716 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.183033 11596 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.186442 11596 ts_tablet_manager.cc:1434] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:49.187297 11596 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:49.192356 11592 tablet_bootstrap.cc:492] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:49.192559 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.192796 11592 ts_tablet_manager.cc:1403] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.000s
W20260709 13:14:49.193933 11365 leader_election.cc:343] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.195430 11592 raft_consensus.cc:359] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.196305 11592 raft_consensus.cc:385] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.196242 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.196698 11592 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.197410 11591 tablet_bootstrap.cc:654] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.197410 11596 tablet_bootstrap.cc:654] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:49.197757 11367 leader_election.cc:343] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.197477 11592 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.198254 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.199059 11870 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.199816 11716 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.200429 11592 ts_tablet_manager.cc:1434] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:49.201299 11592 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:49.207592 11592 tablet_bootstrap.cc:654] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.208802 11596 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.209223 11596 ts_tablet_manager.cc:1403] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 13:14:49.211833 11596 raft_consensus.cc:359] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.212656 11596 raft_consensus.cc:385] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.212950 11596 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.213819 11596 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.216033 11596 ts_tablet_manager.cc:1434] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:14:49.217005 11596 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:49.219372 11591 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.219872 11591 ts_tablet_manager.cc:1403] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.006s
I20260709 13:14:49.222328 11591 raft_consensus.cc:359] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.223119 11591 raft_consensus.cc:385] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.223452 11591 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.224172 11591 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.226116 11591 ts_tablet_manager.cc:1434] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:14:49.227041 11591 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.239118 11596 tablet_bootstrap.cc:654] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.241287 11592 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:49.241758 11592 ts_tablet_manager.cc:1403] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.012s
I20260709 13:14:49.242440 11591 tablet_bootstrap.cc:654] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.245280 11727 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.245754 11727 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.247678 11727 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.249606 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.250937 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:49.251037 11442 leader_election.cc:343] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.251729 11596 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.253082 11596 ts_tablet_manager.cc:1403] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.001s
I20260709 13:14:49.252415 11399 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:49.255528 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.256361 11727 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.256596 11727 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.256939 11727 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.258024 11591 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.257705 11596 raft_consensus.cc:359] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.258353 11596 raft_consensus.cc:385] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.258414 11591 ts_tablet_manager.cc:1403] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.004s
I20260709 13:14:49.258647 11596 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.259359 11596 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.261555 11596 ts_tablet_manager.cc:1434] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:14:49.262662 11596 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:49.244309 11592 raft_consensus.cc:359] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.265105 11592 raft_consensus.cc:385] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.265370 11592 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.266072 11592 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.268191 11592 ts_tablet_manager.cc:1434] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:14:49.269268 11596 tablet_bootstrap.cc:654] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.269057 11727 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.270691 11727 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.271739 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:49.272011 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:49.272275 11399 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:49.273203 11442 leader_election.cc:343] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.275076 11592 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:49.281595 11399 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:49.282908 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.283926 11592 tablet_bootstrap.cc:654] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.289075 11866 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.289608 11866 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:49.290244 11591 raft_consensus.cc:359] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.290416 11866 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.290934 11591 raft_consensus.cc:385] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.291231 11591 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.291795 11591 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.294198 11591 ts_tablet_manager.cc:1434] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 13:14:49.295409 11591 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.300690 11870 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.301367 11870 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.305135 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.305320 11870 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.306646 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.307720 11546 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:49.310395 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.311432 11596 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.311856 11596 ts_tablet_manager.cc:1403] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.004s
I20260709 13:14:49.312018 11591 tablet_bootstrap.cc:654] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.311916 11716 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.318719 11596 raft_consensus.cc:359] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.319396 11596 raft_consensus.cc:385] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.319648 11596 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.322327 11596 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.326390 11596 ts_tablet_manager.cc:1434] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:14:49.328238 11869 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.328895 11869 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.333966 11591 tablet_bootstrap.cc:492] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.334461 11591 ts_tablet_manager.cc:1403] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.005s
I20260709 13:14:49.335842 11869 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.337332 11591 raft_consensus.cc:359] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.337693 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.338039 11591 raft_consensus.cc:385] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.338287 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.338343 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:49.338331 11591 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.338963 11551 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:49.339413 11591 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.340019 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:49.341107 11591 ts_tablet_manager.cc:1434] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:14:49.342059 11591 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.343181 11870 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:49.343679 11870 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.344116 11870 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:49.345046 11596 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:49.347149 11869 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.348192 11591 tablet_bootstrap.cc:654] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.347774 11869 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.351374 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.352166 11473 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:49.353693 11365 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:49.354440 11777 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.354799 11777 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.355149 11777 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.357378 11596 tablet_bootstrap.cc:654] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.362900 11869 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.375481 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.376276 11551 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:49.382907 11861 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.388140 11861 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.386560 11650 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.391656 11650 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.403648 11716 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.404021 11861 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.404938 11870 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.407198 11870 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.420418 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:49.420998 11551 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:49.425447 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:49.426102 11473 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:49.426386 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.427042 11397 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.428341 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:49.429163 11861 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.429515 11861 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.429828 11861 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.417785 11546 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.439298 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.439601 11777 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:49.441048 11516 leader_election.cc:343] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.442690 11861 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.443869 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:14:49.444684 11861 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.445600 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:49.446766 11397 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.447352 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:49.448045 11550 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.430814 11650 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.452195 11650 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.452718 11650 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.453171 11591 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.453711 11591 ts_tablet_manager.cc:1403] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.112s	user 0.031s	sys 0.038s
I20260709 13:14:49.456486 11907 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.457444 11596 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.457103 11907 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.457890 11596 ts_tablet_manager.cc:1403] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.113s	user 0.009s	sys 0.010s
I20260709 13:14:49.454977 11650 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.459699 11650 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.459632 11907 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.464639 11777 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.460774 11596 raft_consensus.cc:359] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.466156 11596 raft_consensus.cc:385] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.466455 11596 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.467175 11596 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.469671 11865 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.469964 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:49.470187 11865 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.469710 11596 ts_tablet_manager.cc:1434] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:14:49.470907 11777 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:49.470964 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:49.472266 11865 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.472373 11477 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.471858 11596 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
W20260709 13:14:49.473912 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54289.8us
I20260709 13:14:49.474794 11551 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:14:49.491343 11591 raft_consensus.cc:359] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.492503 11591 raft_consensus.cc:385] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.492895 11591 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.495654 11592 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:49.496387 11592 ts_tablet_manager.cc:1403] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.222s	user 0.057s	sys 0.050s
I20260709 13:14:49.496888 11591 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.497766 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.498680 11474 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.499131 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.493894 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:49.499789 11475 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.474499 11777 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:49.496033 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.473508 11397 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:49.473817 11650 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.507714 11591 ts_tablet_manager.cc:1434] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.003s
I20260709 13:14:49.504680 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.509646 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.511114 11551 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:49.514479 11869 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.515065 11514 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:49.514935 11869 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.515349 11399 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:49.517024 11869 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:49.506958 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 52658.2us
I20260709 13:14:49.519421 11477 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:49.520506 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.520818 11591 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:49.521325 11545 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:49.522032 11908 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.523180 11908 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:49.524003 11514 leader_election.cc:343] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.524318 11777 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.524622 11514 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:49.524956 11592 raft_consensus.cc:359] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.525597 11861 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.524034 11908 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.527771 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:49.525604 11592 raft_consensus.cc:385] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.529354 11299 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:49.532711 11592 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.532667 11442 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:49.529544 11861 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.536464 11365 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:49.537511 11716 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:49.535077 11865 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.538105 11716 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:49.538412 11865 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.538919 11865 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.539343 11716 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.535122 11592 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.528303 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.552114 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.552793 11397 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.561069 11550 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:49.566517 11592 ts_tablet_manager.cc:1434] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.070s	user 0.003s	sys 0.005s
I20260709 13:14:49.561307 11865 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.564051 11729 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:14:49.572079 11591 tablet_bootstrap.cc:654] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.580118 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.568044 11476 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.576738 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.584053 11861 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:49.589480 11650 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.589680 11861 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.577569 11908 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.600936 11365 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:49.602522 11863 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.595906 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:49.615788 11596 tablet_bootstrap.cc:654] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.625087 11870 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.625761 11870 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.607446 11777 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.598887 11729 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:14:49.603044 11863 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.629648 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:49.634373 11367 leader_election.cc:343] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.630089 11908 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:49.634622 11863 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.634639 11477 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.637791 11514 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:49.637614 11727 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:49.638700 11592 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:49.638844 11908 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:49.632530 11870 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.652941 11908 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.666112 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:49.655663 11865 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:49.668185 11472 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:49.683418 11591 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:49.683908 11591 ts_tablet_manager.cc:1403] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.170s	user 0.019s	sys 0.023s
I20260709 13:14:49.663391 11908 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.676704 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.690654 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:49.714180 11591 raft_consensus.cc:359] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.714723 11591 raft_consensus.cc:385] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:49.714953 11591 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:49.716009 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:49.716645 11551 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:49.721825 11861 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:49.726462 11592 tablet_bootstrap.cc:654] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:49.730607 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:49.731230 11398 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:49.741915 11596 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:49.742599 11596 ts_tablet_manager.cc:1403] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.271s	user 0.015s	sys 0.020s
W20260709 13:14:49.725951 11514 leader_election.cc:343] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:49.739478 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.740610 11861 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:14:49.765353 11299 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 2, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:49.775357 11917 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:14:49.777398 11591 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:49.796888 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:49.800648 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:49.934755 11777 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:49.939802 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:49.940858 11402 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:14:50.045207 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:50.061569 11398 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:50.064056 11908 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.070107 11591 ts_tablet_manager.cc:1434] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.386s	user 0.023s	sys 0.023s
I20260709 13:14:50.070199 11596 raft_consensus.cc:359] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.078325 11596 raft_consensus.cc:385] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.078629 11596 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.090344 11473 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.091203 11591 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:50.093245 11596 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.096330 11596 ts_tablet_manager.cc:1434] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.353s	user 0.006s	sys 0.031s
I20260709 13:14:50.102726 11596 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
W20260709 13:14:50.102349 11365 leader_election.cc:343] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.103899 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:50.106433 11906 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:50.127493 11516 leader_election.cc:343] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.128152 11516 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:50.116227 11908 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:50.135680 11909 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.152697 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:50.154096 11861 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:50.158252 11863 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.157434 11908 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.197685 11592 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:50.198434 11592 ts_tablet_manager.cc:1403] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.568s	user 0.044s	sys 0.051s
I20260709 13:14:50.198009 11596 tablet_bootstrap.cc:654] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:50.190696 11908 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.189316 11863 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.249085 11299 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:50.238406 11442 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:50.267015 11919 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.268075 11908 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:50.270470 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:50.220404 11545 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.243101 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:50.259271 11861 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:50.277956 11919 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.284024 11401 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.293452 11915 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:14:50.275900 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 91958us
I20260709 13:14:50.304489 11591 tablet_bootstrap.cc:654] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:50.299014 11592 raft_consensus.cc:359] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.305140 11592 raft_consensus.cc:385] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.303313 11475 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:50.305437 11592 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.309738 11592 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.309105 11863 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:50.328670 11592 ts_tablet_manager.cc:1434] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.130s	user 0.007s	sys 0.000s
I20260709 13:14:50.334264 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:50.335240 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:50.336815 11909 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.344631 11592 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:50.337381 11909 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.349340 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:50.350718 11909 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:50.360399 11596 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:50.360805 11596 ts_tablet_manager.cc:1403] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.262s	user 0.021s	sys 0.013s
I20260709 13:14:50.344102 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:50.336876 11925 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:14:50.366433 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:50.351439 11926 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260709 13:14:50.369825 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:50.344252 11934 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.061s
I20260709 13:14:50.378201 11777 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:50.320741 11861 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.379782 11922 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:50.386428 11596 raft_consensus.cc:359] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.392999 11596 raft_consensus.cc:385] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.394945 11543 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:50.371388 11472 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:50.393321 11596 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.390125 11298 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:50.400590 11516 leader_election.cc:343] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.448462 11591 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:50.448976 11591 ts_tablet_manager.cc:1403] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.358s	user 0.022s	sys 0.005s
I20260709 13:14:50.450893 11596 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.473066 11932 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.473505 11401 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:50.473907 11932 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.484455 11592 tablet_bootstrap.cc:654] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:50.485129 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:50.485594 11596 ts_tablet_manager.cc:1434] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.125s	user 0.003s	sys 0.016s
I20260709 13:14:50.470186 11869 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:14:50.477524 11932 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:14:50.486178 11442 consensus_peers.cc:597] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.487660 11596 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
W20260709 13:14:50.491732 11442 consensus_peers.cc:597] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:50.492377 11939 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:14:50.524756 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:50.525619 11542 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
W20260709 13:14:50.529150 11942 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:14:50.537398 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:50.544937 11869 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:50.545338 11869 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.545670 11869 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:50.566779 11442 leader_election.cc:343] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.526706 11923 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.067s
I20260709 13:14:50.565816 11591 raft_consensus.cc:359] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.570076 11591 raft_consensus.cc:385] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.570376 11591 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.570840 11591 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.574527 11596 tablet_bootstrap.cc:654] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:50.581233 11592 tablet_bootstrap.cc:492] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:50.581735 11592 ts_tablet_manager.cc:1403] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.237s	user 0.009s	sys 0.009s
I20260709 13:14:50.582147 11591 ts_tablet_manager.cc:1434] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.133s	user 0.011s	sys 0.004s
I20260709 13:14:50.597452 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:50.609854 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:50.611033 11869 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.607183 11933 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:50.616216 11933 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:50.621506 11869 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:50.623975 11861 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.625129 11592 raft_consensus.cc:359] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.626053 11592 raft_consensus.cc:385] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.626397 11592 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.627628 11592 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.617062 11933 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.646917 11592 ts_tablet_manager.cc:1434] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.065s	user 0.003s	sys 0.004s
I20260709 13:14:50.635875 11472 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:50.681093 11592 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:50.688482 11591 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
W20260709 13:14:50.690759 11365 leader_election.cc:343] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:50.700567 11442 leader_election.cc:343] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.701026 11936 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.701107 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:50.701625 11936 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.703063 11866 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.703840 11936 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:50.719278 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:50.720311 11546 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:50.728374 11949 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:14:50.754053 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:50.759624 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:50.763465 11592 tablet_bootstrap.cc:654] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:50.764009 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 49666.9us
I20260709 13:14:50.750393 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:50.772233 11596 tablet_bootstrap.cc:492] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:50.772717 11596 ts_tablet_manager.cc:1403] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.285s	user 0.008s	sys 0.006s
I20260709 13:14:50.763068 11934 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:14:50.769328 11516 leader_election.cc:343] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.773902 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:50.804491 11596 raft_consensus.cc:359] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:50.805207 11596 raft_consensus.cc:385] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:50.805475 11596 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:50.806155 11596 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:50.780683 11514 leader_election.cc:343] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.808962 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:50.811883 11934 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:50.813109 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:50.815390 11596 ts_tablet_manager.cc:1434] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 13:14:50.843317 11546 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
W20260709 13:14:50.843339 11365 leader_election.cc:343] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:50.855648 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:14:50.858696 11957 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:14:50.858764 11922 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:50.860078 11922 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:50.861488 11596 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:50.874308 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:50.878147 11934 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:50.878611 11934 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.878980 11934 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:50.915020 11958 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:14:50.861076 11922 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:50.962858 11960 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:14:50.973768 11591 tablet_bootstrap.cc:654] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:50.997170 11913 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.997747 11913 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.020076 11777 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.029371 11777 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.032059 11777 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.042939 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:51.043697 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:51.113080 11934 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.115782 11934 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.077942 11909 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
W20260709 13:14:51.130586 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 64376.6us
I20260709 13:14:51.148640 11913 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.148799 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:51.149911 11922 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:51.150255 11922 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.150573 11922 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.183679 11592 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.153589 11550 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.184230 11592 ts_tablet_manager.cc:1403] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.505s	user 0.019s	sys 0.036s
I20260709 13:14:51.185853 11596 tablet_bootstrap.cc:654] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.186820 11592 raft_consensus.cc:359] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.187544 11592 raft_consensus.cc:385] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.187834 11592 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.190174 11592 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.194154 11591 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:51.194556 11591 ts_tablet_manager.cc:1403] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.506s	user 0.039s	sys 0.022s
I20260709 13:14:51.197168 11592 ts_tablet_manager.cc:1434] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:14:51.195112 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:51.206779 11592 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:51.208729 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:51.216624 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:51.217376 11551 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:51.218799 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:51.219370 11477 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.228542 11477 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:51.188380 11913 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:14:51.241456 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:51.311602 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47343.3us
W20260709 13:14:51.318395 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 125794us
I20260709 13:14:51.314436 11777 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:51.315653 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 55814.8us
I20260709 13:14:51.323804 11777 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.351325 11971 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:14:51.352533 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:51.383519 11922 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.389556 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44507.9us
I20260709 13:14:51.397493 11592 tablet_bootstrap.cc:654] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.399813 11591 raft_consensus.cc:359] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.342216 11516 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:51.400362 11591 raft_consensus.cc:385] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.352737 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:51.401904 11591 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.404035 11591 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.408538 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66432us
I20260709 13:14:51.409591 11591 ts_tablet_manager.cc:1434] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.215s	user 0.006s	sys 0.001s
I20260709 13:14:51.410066 11908 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:51.410421 11591 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:51.411675 11396 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:14:51.419699 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77215us
I20260709 13:14:51.420881 11933 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:51.424333 11933 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:51.427965 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:51.422880 11777 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.425135 11933 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.446254 11367 leader_election.cc:343] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.446784 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:51.478647 11365 leader_election.cc:343] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.481635 11365 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:51.490125 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:51.492130 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:51.492913 11477 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:51.478178 11922 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.485088 11973 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:51.508352 11365 consensus_peers.cc:597] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.515508 11592 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.516005 11592 ts_tablet_manager.cc:1403] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.310s	user 0.028s	sys 0.003s
I20260709 13:14:51.517020 11591 tablet_bootstrap.cc:654] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.519129 11861 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:51.518277 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:51.520272 11973 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:51.535952 11716 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:51.571209 11592 raft_consensus.cc:359] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.571801 11592 raft_consensus.cc:385] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.572108 11592 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.577816 11596 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:51.578632 11596 ts_tablet_manager.cc:1403] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.718s	user 0.034s	sys 0.028s
I20260709 13:14:51.576190 11908 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:51.578006 11592 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.581359 11596 raft_consensus.cc:359] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.582031 11596 raft_consensus.cc:385] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.582379 11596 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.583079 11596 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.581692 11592 ts_tablet_manager.cc:1434] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.001s
I20260709 13:14:51.588356 11596 ts_tablet_manager.cc:1434] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:14:51.607766 11596 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:51.612743 11592 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:51.626235 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.628419 11298 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.648737 11596 tablet_bootstrap.cc:654] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.651221 11592 tablet_bootstrap.cc:654] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.660419 11591 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:51.661088 11591 ts_tablet_manager.cc:1403] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.251s	user 0.020s	sys 0.024s
I20260709 13:14:51.665769 11591 raft_consensus.cc:359] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.667140 11591 raft_consensus.cc:385] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.667505 11591 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.668243 11591 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.670451 11596 tablet_bootstrap.cc:492] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:51.670840 11592 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.671471 11592 ts_tablet_manager.cc:1403] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.002s
I20260709 13:14:51.672356 11591 ts_tablet_manager.cc:1434] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:51.674813 11596 ts_tablet_manager.cc:1403] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.007s
I20260709 13:14:51.675472 11592 raft_consensus.cc:359] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.678200 11592 raft_consensus.cc:385] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.678507 11592 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.680464 11592 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.680785 11596 raft_consensus.cc:359] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.678875 11591 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:51.678879 11933 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.681437 11596 raft_consensus.cc:385] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.681741 11933 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.683843 11933 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.684603 11592 ts_tablet_manager.cc:1434] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 13:14:51.685837 11592 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:51.686321 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:51.687173 11398 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:51.687554 11591 tablet_bootstrap.cc:654] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.688606 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:51.694165 11592 tablet_bootstrap.cc:654] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.695508 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:14:51.696666 11514 leader_election.cc:343] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.694841 11933 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:51.697937 11933 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.698287 11933 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.703061 11591 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:51.702610 11297 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.703490 11591 ts_tablet_manager.cc:1403] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.005s
I20260709 13:14:51.681735 11596 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.704429 11933 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.704345 11596 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.706549 11596 ts_tablet_manager.cc:1434] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.001s
I20260709 13:14:51.707237 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:51.707561 11596 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:51.707816 11398 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.713552 11596 tablet_bootstrap.cc:654] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.714393 11591 raft_consensus.cc:359] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.715590 11591 raft_consensus.cc:385] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.715940 11591 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.715838 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:51.719551 11909 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.720186 11909 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.716699 11591 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.723574 11514 leader_election.cc:343] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.714402 11398 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:51.724988 11933 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.726060 11591 ts_tablet_manager.cc:1434] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:14:51.726788 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:51.738492 11933 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:51.738858 11591 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:51.739004 11933 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:51.740025 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:51.740695 11475 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:51.740031 11933 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.741526 11909 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.741797 11514 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:51.746652 11909 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:51.748208 11909 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.748560 11909 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.753356 11591 tablet_bootstrap.cc:654] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.754467 11299 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.759305 11919 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.759807 11919 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.761937 11919 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.774664 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:51.776309 11442 leader_election.cc:343] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.775980 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:51.781278 11442 leader_election.cc:343] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.781658 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:51.782140 11596 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:51.782572 11596 ts_tablet_manager.cc:1403] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.075s	user 0.006s	sys 0.031s
I20260709 13:14:51.777817 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:51.787649 11906 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.788132 11906 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.792701 11906 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.788594 11727 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:51.794366 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:51.796545 11596 raft_consensus.cc:359] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.797210 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:51.797925 11919 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:51.798753 11516 leader_election.cc:343] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.798614 11919 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:51.799151 11365 leader_election.cc:343] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.797266 11596 raft_consensus.cc:385] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.800848 11596 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.801323 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:51.802892 11442 leader_election.cc:343] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.802639 11909 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.803143 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:51.803738 11548 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:51.804893 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:51.805652 11727 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:51.805998 11727 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.806310 11727 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.809404 11909 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.809531 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:14:51.811718 11516 leader_election.cc:343] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.813920 11592 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.814410 11592 ts_tablet_manager.cc:1403] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.129s	user 0.010s	sys 0.012s
I20260709 13:14:51.817848 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:51.818323 11543 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.818487 11475 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.801504 11596 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.827373 11545 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:51.824635 11919 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.824254 11297 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.830255 11367 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:51.824889 11475 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:51.831079 11596 ts_tablet_manager.cc:1434] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 13:14:51.831360 11906 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:51.831703 11514 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:51.833650 11906 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.834029 11906 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.834272 11861 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:51.834810 11596 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:51.842726 11906 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.832820 11727 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.834757 11861 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:51.826856 11592 raft_consensus.cc:359] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.843760 11592 raft_consensus.cc:385] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.844034 11592 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.844151 11861 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.844731 11906 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.844650 11592 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.846308 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:51.847273 11592 ts_tablet_manager.cc:1434] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
W20260709 13:14:51.847594 11365 leader_election.cc:343] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.848217 11592 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:51.853722 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:51.854347 11592 tablet_bootstrap.cc:654] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.854308 11543 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.857241 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:14:51.858716 11442 leader_election.cc:343] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.859673 11591 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:51.858915 11297 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.860137 11591 ts_tablet_manager.cc:1403] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.122s	user 0.009s	sys 0.002s
I20260709 13:14:51.861069 11543 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:51.861593 11727 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:51.862210 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:51.863126 11591 raft_consensus.cc:359] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.863790 11591 raft_consensus.cc:385] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.864027 11591 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.864298 11727 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:51.864756 11727 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:51.865475 11727 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.864640 11591 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.868898 11591 ts_tablet_manager.cc:1434] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:14:51.869868 11591 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:51.872524 11596 tablet_bootstrap.cc:654] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.875607 11592 tablet_bootstrap.cc:492] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.876142 11592 ts_tablet_manager.cc:1403] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.001s
I20260709 13:14:51.876325 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:51.876941 11543 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:51.878391 11591 tablet_bootstrap.cc:654] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.878515 11592 raft_consensus.cc:359] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.879215 11592 raft_consensus.cc:385] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.879467 11592 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.880043 11592 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.882134 11592 ts_tablet_manager.cc:1434] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:51.883340 11592 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:51.884225 11297 catalog_manager.cc:5697] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.888944 11592 tablet_bootstrap.cc:654] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.893392 11596 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:51.893860 11596 ts_tablet_manager.cc:1403] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.000s
I20260709 13:14:51.894096 11543 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:51.895316 11367 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:51.896297 11973 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:51.896953 11973 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:51.896762 11596 raft_consensus.cc:359] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.898636 11596 raft_consensus.cc:385] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.898080 11973 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.899025 11596 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.900080 11596 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.901906 11596 ts_tablet_manager.cc:1434] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:51.903028 11596 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:51.909473 11592 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:51.909965 11592 ts_tablet_manager.cc:1403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.000s
I20260709 13:14:51.911664 11591 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:51.912256 11591 ts_tablet_manager.cc:1403] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.005s
I20260709 13:14:51.913496 11596 tablet_bootstrap.cc:654] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:51.915221 11591 raft_consensus.cc:359] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.915880 11591 raft_consensus.cc:385] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.916121 11591 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:51.916702 11591 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.917994 11861 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:51.940250 11591 ts_tablet_manager.cc:1434] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:14:51.964641 11909 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.965369 11909 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.989830 11909 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:51.993098 11592 raft_consensus.cc:359] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:51.990514 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:51.993872 11592 raft_consensus.cc:385] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:51.994278 11592 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
W20260709 13:14:51.995005 11516 leader_election.cc:343] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:51.992563 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:51.995831 11591 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:51.994910 11973 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.995929 11592 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:52.001217 11514 leader_election.cc:343] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.001629 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:52.002298 11908 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.010530 11592 ts_tablet_manager.cc:1434] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.100s	user 0.006s	sys 0.003s
I20260709 13:14:52.013016 11299 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.998525 11973 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.020605 11592 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.020620 11596 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.021417 11596 ts_tablet_manager.cc:1403] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.019s
I20260709 13:14:52.033536 11473 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.034480 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:14:52.035933 11367 leader_election.cc:343] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.037262 11716 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.035655 11596 raft_consensus.cc:359] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.037441 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.037807 11596 raft_consensus.cc:385] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.037730 11716 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.038587 11596 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
W20260709 13:14:52.038753 11365 leader_election.cc:343] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.039172 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.041595 11923 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.039387 11596 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.048512 11596 ts_tablet_manager.cc:1434] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.004s
I20260709 13:14:52.054895 11861 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.042013 11973 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.057323 11716 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.044319 11909 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.060072 11591 tablet_bootstrap.cc:654] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.071256 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.072052 11548 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:52.072127 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.074661 11596 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.061190 11909 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.080772 11365 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.086648 11923 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.090245 11592 tablet_bootstrap.cc:654] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.091540 11909 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:52.094100 11923 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.096277 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:14:52.098837 11516 leader_election.cc:343] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.105420 11596 tablet_bootstrap.cc:654] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.120810 11591 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.121336 11591 ts_tablet_manager.cc:1403] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.133s	user 0.008s	sys 0.009s
I20260709 13:14:52.122067 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.122841 11471 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:52.124560 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:52.126210 11591 raft_consensus.cc:359] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.126904 11591 raft_consensus.cc:385] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.127192 11591 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.127413 11909 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.127821 11909 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.128239 11909 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.127867 11591 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.130949 11591 ts_tablet_manager.cc:1434] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:14:52.135774 11591 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.140691 11592 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.145107 11592 ts_tablet_manager.cc:1403] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.005s
I20260709 13:14:52.147567 11592 raft_consensus.cc:359] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.148057 11592 raft_consensus.cc:385] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.148274 11592 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.148671 11909 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.148839 11592 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.151764 11592 ts_tablet_manager.cc:1434] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:14:52.151696 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:14:52.153097 11516 leader_election.cc:343] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.154145 11591 tablet_bootstrap.cc:654] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.155689 11592 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.158265 11909 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:52.159260 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.160794 11596 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.167980 11596 ts_tablet_manager.cc:1403] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.002s
I20260709 13:14:52.163066 11592 tablet_bootstrap.cc:654] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.167379 11473 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.181182 11596 raft_consensus.cc:359] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.181752 11596 raft_consensus.cc:385] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.182014 11596 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.182607 11596 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.184095 11596 ts_tablet_manager.cc:1434] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:14:52.185034 11596 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.187776 11591 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.188273 11591 ts_tablet_manager.cc:1403] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.004s
I20260709 13:14:52.190809 11591 raft_consensus.cc:359] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.193212 11591 raft_consensus.cc:385] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.193488 11591 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.195240 11592 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.195720 11592 ts_tablet_manager.cc:1403] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.002s
I20260709 13:14:52.194134 11591 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.199570 11591 ts_tablet_manager.cc:1434] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:14:52.201257 11591 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.201180 11592 raft_consensus.cc:359] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.201074 11473 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:52.202183 11592 raft_consensus.cc:385] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.202451 11592 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.202947 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:14:52.203075 11592 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.204074 11909 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.204690 11909 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:52.204993 11596 tablet_bootstrap.cc:654] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.205402 11592 ts_tablet_manager.cc:1434] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:14:52.205466 11909 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.206687 11592 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.211860 11591 tablet_bootstrap.cc:654] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.213336 11908 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.213845 11908 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.215021 11592 tablet_bootstrap.cc:654] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.215953 11908 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:52.216645 11923 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.217099 11923 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.218979 11596 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.219460 11596 ts_tablet_manager.cc:1403] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.004s
I20260709 13:14:52.219497 11923 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.219318 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.221503 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.221848 11396 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:52.223179 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:14:52.224300 11367 leader_election.cc:343] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.225564 11908 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.226033 11908 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.226451 11908 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.230418 11973 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.230837 11591 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.231314 11591 ts_tablet_manager.cc:1403] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 13:14:52.230962 11973 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.226338 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.233361 11475 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 0.
I20260709 13:14:52.234400 11591 raft_consensus.cc:359] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.235162 11591 raft_consensus.cc:385] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.235442 11591 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.236053 11591 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.234887 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.238057 11591 ts_tablet_manager.cc:1434] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:52.227579 11592 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.228302 11596 raft_consensus.cc:359] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.238586 11596 raft_consensus.cc:385] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.238922 11596 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
W20260709 13:14:52.239315 11365 leader_election.cc:343] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.239804 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.238610 11592 ts_tablet_manager.cc:1403] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 13:14:52.240721 11716 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.239549 11596 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.242244 11592 raft_consensus.cc:359] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.242673 11596 ts_tablet_manager.cc:1434] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.002s
I20260709 13:14:52.242820 11592 raft_consensus.cc:385] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.243160 11592 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.243757 11596 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.243746 11592 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.244071 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.245846 11592 ts_tablet_manager.cc:1434] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.001s
W20260709 13:14:52.246599 11365 leader_election.cc:343] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.247206 11592 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.251823 11596 tablet_bootstrap.cc:654] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.252619 11908 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.255126 11973 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.255298 11908 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:52.255641 11591 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.255720 11592 tablet_bootstrap.cc:654] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.256608 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.257228 11548 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:52.259531 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:52.259673 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.260481 11473 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.261682 11591 tablet_bootstrap.cc:654] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.262234 11923 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.262640 11923 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.262974 11923 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.271497 11473 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:52.272048 11956 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.272634 11591 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.272521 11956 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.273089 11591 ts_tablet_manager.cc:1403] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:14:52.274535 11956 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.276371 11591 raft_consensus.cc:359] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.277029 11591 raft_consensus.cc:385] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.277272 11591 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.277626 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:52.277894 11591 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.278328 11908 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.278752 11908 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:52.279942 11591 ts_tablet_manager.cc:1434] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 13:14:52.281047 11514 consensus_peers.cc:597] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:52.279551 11908 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.284612 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.290684 11906 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.293311 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.293515 11906 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.294003 11399 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:52.280862 11591 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.295475 11906 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.296499 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.297187 11473 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
I20260709 13:14:52.298281 11365 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.299752 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.299042 11906 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.300478 11906 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.300925 11906 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:52.305094 11367 leader_election.cc:343] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.305832 11591 tablet_bootstrap.cc:654] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.315378 11906 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:52.319911 11442 leader_election.cc:343] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.322326 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:52.326956 11395 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.330539 11906 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.331635 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.372565 11475 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:52.370371 11365 consensus_peers.cc:597] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:52.370854 11906 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.363564 11919 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.377702 11919 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.380390 11919 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.392655 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:52.400563 11399 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.404819 11861 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:52.419831 11367 leader_election.cc:343] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.380642 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:14:52.425344 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44795.3us
I20260709 13:14:52.426260 11951 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.426666 11951 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.427094 11951 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.443197 11923 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.448264 11951 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.449226 11298 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.503436 11298 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.452607 11861 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.456943 11919 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.464051 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.483739 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.508112 11395 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:14:52.532227 11475 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:52.526005 11973 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.539736 11973 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.513530 11919 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.571758 11596 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.572417 11596 ts_tablet_manager.cc:1403] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.329s	user 0.029s	sys 0.039s
I20260709 13:14:52.571061 12026 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.576529 11919 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.594080 11951 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.597827 11591 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.598361 11591 ts_tablet_manager.cc:1403] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.318s	user 0.071s	sys 0.136s
I20260709 13:14:52.579295 12026 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.601150 12026 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.603933 11596 raft_consensus.cc:359] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.604630 11596 raft_consensus.cc:385] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.604895 11596 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.607122 11973 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.609827 11923 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.621011 11591 raft_consensus.cc:359] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.621474 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.621832 11591 raft_consensus.cc:385] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.622200 11591 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.622794 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:52.623386 11544 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.623584 11365 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:14:52.625200 11348 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:52.625638 11906 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.626156 11906 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:52.626945 11906 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.633208 11596 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.635473 11596 ts_tablet_manager.cc:1434] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 13:14:52.637167 11596 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.638397 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.639127 11475 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:52.640329 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.644241 11544 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:14:52.645376 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:52.646289 11959 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.646718 11959 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:52.647517 11959 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.648125 11591 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:52.654208 11516 consensus_peers.cc:597] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:52.654956 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:14:52.656495 11365 leader_election.cc:343] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.658159 11906 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:52.662478 11906 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:14:52.663162 11906 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.666563 11592 tablet_bootstrap.cc:492] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.667049 11592 ts_tablet_manager.cc:1403] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.420s	user 0.114s	sys 0.178s
I20260709 13:14:52.668447 11596 tablet_bootstrap.cc:654] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.669884 11591 ts_tablet_manager.cc:1434] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:14:52.669648 11592 raft_consensus.cc:359] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.670358 11592 raft_consensus.cc:385] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.671094 11591 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.685314 11592 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
W20260709 13:14:52.682242 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 96627.2us
I20260709 13:14:52.688710 11592 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.690920 11592 ts_tablet_manager.cc:1434] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.001s
I20260709 13:14:52.691903 11592 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.694087 11591 tablet_bootstrap.cc:654] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:52.696766 11442 leader_election.cc:343] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.709899 11591 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.710352 11591 ts_tablet_manager.cc:1403] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.003s
I20260709 13:14:52.712759 11591 raft_consensus.cc:359] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.713392 11591 raft_consensus.cc:385] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.713656 11591 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.717733 11596 tablet_bootstrap.cc:492] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.718179 11596 ts_tablet_manager.cc:1403] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.081s	user 0.006s	sys 0.013s
I20260709 13:14:52.720750 11596 raft_consensus.cc:359] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.722393 11596 raft_consensus.cc:385] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.722662 11596 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.727310 11908 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.728263 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.728956 11395 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:52.728878 11906 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.729607 11592 tablet_bootstrap.cc:654] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.730142 11398 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:52.724195 11596 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.732137 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.732671 11475 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.733209 11591 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.732749 11596 ts_tablet_manager.cc:1434] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.005s
I20260709 13:14:52.734767 11596 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.734944 11936 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:52.736093 11936 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.736549 11936 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:14:52.737470 11936 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.738889 11936 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.740612 11596 tablet_bootstrap.cc:654] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.742727 11475 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:14:52.744129 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.744911 11959 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.745339 11959 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:52.746105 11959 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.751436 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:52.754832 11906 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.760316 11591 ts_tablet_manager.cc:1434] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.001s
I20260709 13:14:52.761325 11591 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.767627 11592 tablet_bootstrap.cc:492] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.768107 11592 ts_tablet_manager.cc:1403] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.027s
I20260709 13:14:52.768329 11591 tablet_bootstrap.cc:654] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.770229 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.770852 11542 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:52.773561 11592 raft_consensus.cc:359] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.774237 11592 raft_consensus.cc:385] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.774505 11592 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.774535 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:52.775189 11395 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.775519 11592 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.777515 11592 ts_tablet_manager.cc:1434] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:14:52.778430 11592 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Bootstrap starting.
I20260709 13:14:52.779932 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:52.785202 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.785805 11398 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:52.787043 11592 tablet_bootstrap.cc:654] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.788578 11596 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.789080 11596 ts_tablet_manager.cc:1403] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 13:14:52.790335 11395 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:52.791946 11592 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: No bootstrap required, opened a new log
I20260709 13:14:52.791836 11596 raft_consensus.cc:359] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.791987 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.792541 11592 ts_tablet_manager.cc:1403] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 13:14:52.792593 11596 raft_consensus.cc:385] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.792796 11545 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:52.793007 11596 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.793452 11591 tablet_bootstrap.cc:492] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.793939 11591 ts_tablet_manager.cc:1403] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.009s
I20260709 13:14:52.793658 11596 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.795280 11592 raft_consensus.cc:359] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.795934 11592 raft_consensus.cc:385] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.796209 11592 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.796896 11591 raft_consensus.cc:359] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.796901 11596 ts_tablet_manager.cc:1434] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:52.798044 11591 raft_consensus.cc:385] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.798453 11591 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.799120 11596 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.799163 11591 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.797506 11592 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.801506 11591 ts_tablet_manager.cc:1434] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:14:52.802484 11591 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Bootstrap starting.
I20260709 13:14:52.805583 11908 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.807518 11592 ts_tablet_manager.cc:1434] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:14:52.805931 11596 tablet_bootstrap.cc:654] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.808213 11591 tablet_bootstrap.cc:654] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.811277 11544 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:14:52.817607 11908 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.818130 11908 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.821110 11908 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:52.821121 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:52.822378 11398 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:52.834374 11591 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: No bootstrap required, opened a new log
I20260709 13:14:52.834954 11591 ts_tablet_manager.cc:1403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.004s
I20260709 13:14:52.837639 11591 raft_consensus.cc:359] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.838442 11591 raft_consensus.cc:385] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.838763 11591 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.840691 11475 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.849349 11591 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.864967 11959 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.865725 11959 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.868465 11959 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.869017 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.869359 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.869668 11471 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 0.
W20260709 13:14:52.870528 11367 leader_election.cc:343] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.871989 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.873200 11716 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.873531 11716 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.873867 11716 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.880541 11716 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.882972 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:14:52.883558 11470 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.890112 11470 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
W20260709 13:14:52.892194 11442 leader_election.cc:343] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.893296 11591 ts_tablet_manager.cc:1434] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.003s
I20260709 13:14:52.905431 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.905773 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.906304 12031 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.904389 11716 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.906503 11906 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.907405 11906 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:14:52.908221 11906 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.908286 12028 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:52.910019 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.910772 12031 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.911513 12031 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.912017 12031 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.921561 12028 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.922080 12028 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.924099 12028 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.910816 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.892961 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:52.926028 12028 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:52.926367 12028 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.926713 12028 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:52.928782 11367 leader_election.cc:343] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:52.929486 12031 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.931895 12031 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.934536 12028 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.945003 11596 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:52.945525 11596 ts_tablet_manager.cc:1403] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.147s	user 0.042s	sys 0.076s
I20260709 13:14:52.945286 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:52.946688 12028 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.947521 12027 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:52.948159 12027 raft_consensus.cc:697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:52.949276 12027 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.951325 11959 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.951777 11959 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.950675 11298 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:52.953852 11959 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:52.948175 11596 raft_consensus.cc:359] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.954730 11596 raft_consensus.cc:385] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:52.954093 11298 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:52.954939 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:52.955080 11596 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:52.955963 11596 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:52.956615 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.956461 11298 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:52.957337 11470 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 1: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:14:52.958361 11596 ts_tablet_manager.cc:1434] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:14:52.959040 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:52.960009 11959 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.960423 11596 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Bootstrap starting.
I20260709 13:14:52.965698 11471 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:14:52.968271 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:52.969826 11596 tablet_bootstrap.cc:654] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:52.978107 11475 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:52.985073 11959 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:52.995599 11909 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.996311 11861 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:52.999509 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:53.000927 11861 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.002061 12028 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.005525 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:53.007120 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:53.007896 11544 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:53.009572 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:53.010128 11542 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.011140 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:53.011958 11543 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:53.013203 11442 leader_election.cc:343] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:53.024385 11298 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.025851 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:53.027832 11395 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.024873 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:53.034545 11545 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.040016 11542 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:53.034852 11398 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.041145 11300 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.053078 11906 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.070596 11908 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.071126 11908 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.073150 11908 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:53.094692 11861 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:53.105417 11398 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:53.111259 11477 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.118779 11906 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.125357 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:53.126394 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:53.128785 11925 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:53.130031 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:14:53.131481 11919 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:53.132037 11919 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:53.132887 11919 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.136574 11543 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:53.139559 11395 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:53.141433 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:53.142267 11473 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:14:53.148489 11544 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:53.149770 11959 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:53.176235 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:53.193005 12031 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:14:53.193519 12031 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.197214 12031 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:53.203851 11956 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:53.208213 11956 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:53.209201 11956 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.209764 11473 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:53.212705 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:53.230106 11861 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.220685 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:53.231755 11973 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:53.233596 11973 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:53.238087 11861 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:53.246567 11926 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.247138 11926 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.249055 11926 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:53.256043 11906 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:53.258085 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:53.263254 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:14:53.270742 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:53.271490 11395 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 0.
I20260709 13:14:53.280108 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:53.282274 12028 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:53.283401 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:14:53.285326 12028 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:53.285733 12028 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.286123 12028 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.304162 11596 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: No bootstrap required, opened a new log
I20260709 13:14:53.304831 12028 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:14:53.306205 11442 leader_election.cc:343] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:53.307308 12028 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:53.312225 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:14:53.304884 11596 ts_tablet_manager.cc:1403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent bootstrapping tablet: real 0.345s	user 0.113s	sys 0.144s
I20260709 13:14:53.312371 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:14:53.313979 11395 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.315465 11596 raft_consensus.cc:359] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.316155 11596 raft_consensus.cc:385] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:53.316406 11596 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Initialized, Role: FOLLOWER
I20260709 13:14:53.317028 11596 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.319196 11596 ts_tablet_manager.cc:1434] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
W20260709 13:14:53.320025 11442 leader_election.cc:343] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:53.321235 11395 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:14:53.322564 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:14:53.323272 11956 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:53.323743 11956 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:14:53.324504 11956 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.333576 11300 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.335060 11300 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.358932 11299 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.376727 11956 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.387681 11919 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.427796 11933 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.428218 11933 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:14:53.430186 11933 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:53.430900 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:14:53.432652 11395 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 1: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:14:53.432477 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:14:53.435406 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:14:53.436138 11933 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:53.442860 11933 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:53.451996 11959 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.463366 11959 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.581017 10252 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:53.589179 12071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:53.591496 12072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:53.593222 11919 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.593973 10252 server_base.cc:1061] running on GCE node
W20260709 13:14:53.595767 12074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:53.597232 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:53.598304 10252 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:53.598632 10252 hybrid_clock.cc:648] HybridClock initialized: now 1783602893598611 us; error 0 us; skew 500 ppm
I20260709 13:14:53.599457 10252 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:53.605824 11925 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.608987 10252 webserver.cc:533] Webserver started at http://127.10.3.4:40327/ using document root <none> and password file <none>
I20260709 13:14:53.613325 10252 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:53.613613 10252 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:53.614092 10252 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:53.619419 10252 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-3-root/instance:
uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
format_stamp: "Formatted at 2026-07-09 13:14:53 on dist-test-slave-gvjr"
I20260709 13:14:53.631089 10252 fs_manager.cc:696] Time spent creating directory manager: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:14:53.639977 11925 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.659085 12082 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:53.661409 10252 fs_manager.cc:730] Time spent opening block manager: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:14:53.661895 10252 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-3-root
uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
format_stamp: "Formatted at 2026-07-09 13:14:53 on dist-test-slave-gvjr"
I20260709 13:14:53.662349 10252 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:53.664422 11951 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.694322 10252 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:53.697762 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:53.703040 10252 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:53.706115 10252 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:53.706354 10252 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:53.706646 10252 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:53.706840 10252 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:53.769881 11919 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.770627 11925 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:53.889292 10252 rpc_server.cc:307] RPC server started. Bound to: 127.10.3.4:45993
I20260709 13:14:53.889375 12147 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.3.4:45993 every 8 connection(s)
I20260709 13:14:53.894666 10252 auto_leader_rebalancer-test.cc:384] add a tserver: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:53.908648 12148 heartbeater.cc:344] Connected to a master server at 127.10.3.62:34815
I20260709 13:14:53.909097 12148 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:53.909942 12148 heartbeater.cc:507] Master 127.10.3.62:34815 requested a full tablet report, sending...
I20260709 13:14:53.911959 11298 ts_manager.cc:194] Registered new tserver with Master: b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:14:53.913350 11298 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38378
I20260709 13:14:54.774614 11398 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:54.780642 11548 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.782884 12153 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.783543 11477 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.787308 11906 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.802774 11906 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.805842 11548 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.820623 11398 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:54.825484 11298 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:54.812059 11477 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:14:54.837464 11366 consensus_peers.cc:597] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff56d6db2ff44eb19771b9166c042077. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:54.851809 11366 consensus_peers.cc:597] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfd1f78aa1d843c28021b808cf5fd960. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:54.859107 11477 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.859591 11548 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.861035 11906 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.873934 12164 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:54.883175 12164 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.884179 11477 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.885728 11542 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.910035 11300 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.915665 12148 heartbeater.cc:499] Master 127.10.3.62:34815 was elected leader, sending a full tablet report...
I20260709 13:14:54.917183 11477 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:54.925791 11542 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.927945 11398 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.929481 11956 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.930394 12161 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:54.939996 11441 consensus_peers.cc:597] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a11e4197bc8843c69871c3f11dc98474. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:54.953003 12161 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.959561 11546 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.961162 11398 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:54.969208 11477 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:54.970419 11299 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:54.988684 11441 consensus_peers.cc:597] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 778170f25cb24e08b57cbbc852e8a007. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:54.989540 11542 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.989713 11398 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:54.994256 12161 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.995229 11956 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.004776 11956 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:55.006487 11546 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:55.009238 11398 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:55.022902 11300 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:55.309428 12179 ts_tablet_manager.cc:933] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:55.313139 12179 tablet_copy_client.cc:323] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:14:55.325826 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet a11e4197bc8843c69871c3f11dc98474 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:14:55.326422 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet a11e4197bc8843c69871c3f11dc98474 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-a11e4197bc8843c69871c3f11dc98474
I20260709 13:14:55.333616 11487 tablet_copy_source_session.cc:215] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:55.337337 12179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a11e4197bc8843c69871c3f11dc98474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:55.349401 12179 tablet_copy_client.cc:806] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:55.350055 12179 tablet_copy_client.cc:670] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:55.355504 12179 tablet_copy_client.cc:538] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:55.382445 12179 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:55.420269 12184 ts_tablet_manager.cc:933] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:14:55.428038 12182 ts_tablet_manager.cc:933] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:55.428500 12184 tablet_copy_client.cc:323] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:14:55.439445 12182 tablet_copy_client.cc:323] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:14:55.440688 12179 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:55.442673 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 778170f25cb24e08b57cbbc852e8a007 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:14:55.443241 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 778170f25cb24e08b57cbbc852e8a007 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-778170f25cb24e08b57cbbc852e8a007
I20260709 13:14:55.445817 12179 tablet_bootstrap.cc:492] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:55.446594 12179 ts_tablet_manager.cc:1403] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.023s
I20260709 13:14:55.449522 12179 raft_consensus.cc:359] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.450344 12179 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:55.450498 11487 tablet_copy_source_session.cc:215] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:55.451112 12179 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.452179 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet ff56d6db2ff44eb19771b9166c042077 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:14:55.452741 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet ff56d6db2ff44eb19771b9166c042077 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-ff56d6db2ff44eb19771b9166c042077
I20260709 13:14:55.461480 12179 ts_tablet_manager.cc:1434] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.007s
I20260709 13:14:55.463353 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-a11e4197bc8843c69871c3f11dc98474 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:14:55.463837 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-a11e4197bc8843c69871c3f11dc98474 on tablet a11e4197bc8843c69871c3f11dc98474 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:55.466307 12182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778170f25cb24e08b57cbbc852e8a007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:55.471515 11412 tablet_copy_source_session.cc:215] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:55.474509 12179 ts_tablet_manager.cc:933] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:14:55.475504 12184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff56d6db2ff44eb19771b9166c042077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:55.492081 12184 tablet_copy_client.cc:806] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:55.492184 12182 tablet_copy_client.cc:806] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:55.492671 12184 tablet_copy_client.cc:670] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:55.492791 12182 tablet_copy_client.cc:670] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:55.492113 12179 tablet_copy_client.cc:323] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:14:55.501482 12182 tablet_copy_client.cc:538] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:55.506268 11411 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet cfd1f78aa1d843c28021b808cf5fd960 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:14:55.506922 11411 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet cfd1f78aa1d843c28021b808cf5fd960 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-cfd1f78aa1d843c28021b808cf5fd960
I20260709 13:14:55.509714 12182 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:55.514493 11411 tablet_copy_source_session.cc:215] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:55.516649 12184 tablet_copy_client.cc:538] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:55.517976 12179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd1f78aa1d843c28021b808cf5fd960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:55.540733 12184 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:55.543260 12182 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:55.544202 12182 tablet_bootstrap.cc:492] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:55.544857 12182 ts_tablet_manager.cc:1403] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.004s
I20260709 13:14:55.545528 12179 tablet_copy_client.cc:806] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:55.546144 12179 tablet_copy_client.cc:670] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:55.548022 12182 raft_consensus.cc:359] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.548882 12182 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:55.549511 12182 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.551790 12182 ts_tablet_manager.cc:1434] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:55.555845 12179 tablet_copy_client.cc:538] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:55.564649 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-778170f25cb24e08b57cbbc852e8a007 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:14:55.565168 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-778170f25cb24e08b57cbbc852e8a007 on tablet 778170f25cb24e08b57cbbc852e8a007 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:55.567080 12179 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:55.585016 12184 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:55.585855 12184 tablet_bootstrap.cc:492] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:55.586977 12184 ts_tablet_manager.cc:1403] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.003s
I20260709 13:14:55.588109 12179 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:55.588972 12179 tablet_bootstrap.cc:492] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:55.589620 12179 ts_tablet_manager.cc:1403] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 13:14:55.592811 12184 raft_consensus.cc:359] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.593598 12184 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:55.593372 12179 raft_consensus.cc:359] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.594276 12179 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:55.594332 12184 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.594967 12179 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:55.596652 12184 ts_tablet_manager.cc:1434] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:14:55.598088 12179 ts_tablet_manager.cc:1434] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:14:55.600404 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-cfd1f78aa1d843c28021b808cf5fd960 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:14:55.600817 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-cfd1f78aa1d843c28021b808cf5fd960 on tablet cfd1f78aa1d843c28021b808cf5fd960 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:55.601748 11411 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-ff56d6db2ff44eb19771b9166c042077 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:14:55.602139 11411 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-ff56d6db2ff44eb19771b9166c042077 on tablet ff56d6db2ff44eb19771b9166c042077 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:55.693517 12123 raft_consensus.cc:1217] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:55.881343 12123 raft_consensus.cc:1217] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:55.891131 12123 raft_consensus.cc:1217] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:55.949213 12123 raft_consensus.cc:1217] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:56.302812 12162 raft_consensus.cc:1064] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:56.310375 12162 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.319553 12197 raft_consensus.cc:1064] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:56.323450 12123 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.324007 12197 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.326467 12164 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.331395 11542 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.332949 12195 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.357643 11468 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.359473 11906 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.372298 12192 raft_consensus.cc:1064] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:56.377107 12192 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.419502 12123 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.420542 12122 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.422168 12197 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:56.422228 12192 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.424549 11398 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.425396 11397 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.427153 11546 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.431133 11548 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.432341 11956 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:14:56.438418 12161 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.440012 12197 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:56.440024 12206 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:14:56.491336 12195 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:14:56.577813 12203 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:14:56.552090 11468 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.662824 12122 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.687119 11300 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.689623 12206 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:14:56.691277 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 112094us
I20260709 13:14:56.691884 12200 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.714002 12123 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.718838 11906 raft_consensus.cc:1064] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:56.721323 11906 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.743532 11542 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.778939 11543 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.782913 11471 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.783165 12122 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.784405 11906 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:56.785513 12219 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.792328 11548 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.807837 11546 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:56.810002 12219 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:56.814899 11398 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.817864 11396 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.822218 12123 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.884920 12122 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.883618 12220 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.887633 11475 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.916957 11545 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.924264 11396 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.914511 11298 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.933869 11298 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.936029 11298 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.938947 12122 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:56.942334 11475 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:56.945070 12216 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:56.947077 12164 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:56.957275 12220 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.961333 12122 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.970029 11471 raft_consensus.cc:2955] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.978197 11397 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.982894 11298 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:56.986171 11475 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.984086 11468 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:56.997951 12123 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:56.998232 11546 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.000181 12200 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:57.001047 11546 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.003098 12122 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.004745 12192 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:57.007372 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cfd1f78aa1d843c28021b808cf5fd960 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:57.008251 12206 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.012985 12197 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:57.014134 12200 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:57.018978 12122 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.019182 12197 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.025154 11546 raft_consensus.cc:2955] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.027325 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet cfd1f78aa1d843c28021b808cf5fd960 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:14:57.029254 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a11e4197bc8843c69871c3f11dc98474 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:57.031029 12123 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.032745 12119 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.034487 12208 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:57.034395 11543 raft_consensus.cc:2955] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.045511 12164 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.051884 11299 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.059988 11299 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.065855 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff56d6db2ff44eb19771b9166c042077 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:57.053527 12231 tablet_replica.cc:333] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:14:57.054494 11548 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.056419 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 778170f25cb24e08b57cbbc852e8a007 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:57.056833 12123 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.068852 12208 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:57.073041 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet a11e4197bc8843c69871c3f11dc98474 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:14:57.075040 11383 tablet_service.cc:1558] Processing DeleteTablet for tablet 778170f25cb24e08b57cbbc852e8a007 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:14:57.092811 12231 raft_consensus.cc:2243] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.091768 11298 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.093652 12231 raft_consensus.cc:2272] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.093822 12232 tablet_replica.cc:333] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:14:57.109249 12231 ts_tablet_manager.cc:1916] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:57.109083 11546 raft_consensus.cc:2955] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:57.115688 12232 raft_consensus.cc:2243] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.117379 12232 raft_consensus.cc:2272] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.119357 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet ff56d6db2ff44eb19771b9166c042077 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:14:57.122964 12232 ts_tablet_manager.cc:1916] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:57.123111 12233 tablet_replica.cc:333] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:14:57.124323 12233 raft_consensus.cc:2243] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.125075 12233 raft_consensus.cc:2272] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.128424 12233 ts_tablet_manager.cc:1916] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:57.140776 12231 ts_tablet_manager.cc:1929] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:57.141465 12231 log.cc:1199] T cfd1f78aa1d843c28021b808cf5fd960 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/cfd1f78aa1d843c28021b808cf5fd960
I20260709 13:14:57.143362 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet cfd1f78aa1d843c28021b808cf5fd960 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:14:57.144953 12232 ts_tablet_manager.cc:1929] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:57.145318 12232 log.cc:1199] T a11e4197bc8843c69871c3f11dc98474 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/a11e4197bc8843c69871c3f11dc98474
I20260709 13:14:57.146839 12232 tablet_replica.cc:333] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:14:57.146725 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet a11e4197bc8843c69871c3f11dc98474 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:14:57.147851 12232 raft_consensus.cc:2243] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.148205 12233 ts_tablet_manager.cc:1929] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:57.148416 12232 raft_consensus.cc:2272] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.148604 12233 log.cc:1199] T ff56d6db2ff44eb19771b9166c042077 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/ff56d6db2ff44eb19771b9166c042077
I20260709 13:14:57.150170 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet ff56d6db2ff44eb19771b9166c042077 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:14:57.151548 12232 ts_tablet_manager.cc:1916] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:57.162966 12232 ts_tablet_manager.cc:1929] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:57.163350 12232 log.cc:1199] T 778170f25cb24e08b57cbbc852e8a007 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/778170f25cb24e08b57cbbc852e8a007
I20260709 13:14:57.164517 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 778170f25cb24e08b57cbbc852e8a007 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:14:58.207015 11471 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:14:58.213428 11441 consensus_peers.cc:597] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dba44ee610b54495a5f1cfea0a876524. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:58.214069 11397 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.214107 11543 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.215428 12214 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.216168 12206 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.223718 12206 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.225453 11543 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.227638 11397 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.247929 11297 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.290127 11543 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.298714 11471 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.300050 12217 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.303412 11397 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.306173 12198 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.324229 12198 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.335765 11471 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:14:58.347044 11515 consensus_peers.cc:597] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b150719d534f439aa33eb70af414a5d0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:58.348230 11397 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.358577 11543 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.362613 11298 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.376752 11397 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.378188 12198 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.382121 11471 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.384270 12217 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:58.382884 11515 consensus_peers.cc:597] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6454d815ae44cdda2fe8d013fca9b4b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:58.413251 12248 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.414794 11397 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.417477 11468 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.430830 11398 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.432219 11297 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.450116 11548 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:58.452091 11366 consensus_peers.cc:597] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e8ed42aaa39460bbea1f8deeccd0adc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:58.452571 12208 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.455178 11468 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:58.456689 12216 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.463966 12208 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.468274 11546 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.478214 11471 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:14:58.490892 11298 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.627440 12257 ts_tablet_manager.cc:933] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:14:58.629735 12257 tablet_copy_client.cc:323] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:14:58.631830 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet dba44ee610b54495a5f1cfea0a876524 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:14:58.632406 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet dba44ee610b54495a5f1cfea0a876524 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-dba44ee610b54495a5f1cfea0a876524
I20260709 13:14:58.640307 11487 tablet_copy_source_session.cc:215] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.643759 12257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dba44ee610b54495a5f1cfea0a876524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.659354 12257 tablet_copy_client.cc:806] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.659945 12257 tablet_copy_client.cc:670] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.665607 12257 tablet_copy_client.cc:538] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.675742 12257 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:58.703480 12257 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:58.704432 12257 tablet_bootstrap.cc:492] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:58.705103 12257 ts_tablet_manager.cc:1403] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.007s
I20260709 13:14:58.709915 12257 raft_consensus.cc:359] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.710764 12257 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:58.711745 12260 ts_tablet_manager.cc:933] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:58.711460 12257 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.719640 12257 ts_tablet_manager.cc:1434] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.014s	user 0.013s	sys 0.001s
I20260709 13:14:58.721490 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-dba44ee610b54495a5f1cfea0a876524 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:14:58.721978 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-dba44ee610b54495a5f1cfea0a876524 on tablet dba44ee610b54495a5f1cfea0a876524 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:58.725716 12260 tablet_copy_client.cc:323] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:14:58.742028 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet b150719d534f439aa33eb70af414a5d0 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:14:58.742609 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet b150719d534f439aa33eb70af414a5d0 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-b150719d534f439aa33eb70af414a5d0
I20260709 13:14:58.748531 11561 tablet_copy_source_session.cc:215] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.752619 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b150719d534f439aa33eb70af414a5d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.764873 12260 tablet_copy_client.cc:806] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.765448 12260 tablet_copy_client.cc:670] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.770036 12260 tablet_copy_client.cc:538] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.777132 12260 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:58.795889 12260 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:58.796619 12260 tablet_bootstrap.cc:492] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:58.797181 12260 ts_tablet_manager.cc:1403] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:14:58.800176 12260 raft_consensus.cc:359] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.800973 12260 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:58.801604 12260 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.804212 12260 ts_tablet_manager.cc:1434] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:14:58.805855 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b150719d534f439aa33eb70af414a5d0 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:14:58.806267 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b150719d534f439aa33eb70af414a5d0 on tablet b150719d534f439aa33eb70af414a5d0 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:58.897462 12260 ts_tablet_manager.cc:933] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:14:58.899164 12260 tablet_copy_client.cc:323] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:14:58.901135 11411 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 5e8ed42aaa39460bbea1f8deeccd0adc from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:14:58.901678 11411 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 5e8ed42aaa39460bbea1f8deeccd0adc from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-5e8ed42aaa39460bbea1f8deeccd0adc
I20260709 13:14:58.907670 11411 tablet_copy_source_session.cc:215] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.910621 12260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e8ed42aaa39460bbea1f8deeccd0adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:58.925647 12260 tablet_copy_client.cc:806] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:58.926533 12260 tablet_copy_client.cc:670] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:58.932171 12260 tablet_copy_client.cc:538] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:58.941864 12260 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:58.962111 12257 ts_tablet_manager.cc:933] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:14:58.966792 12257 tablet_copy_client.cc:323] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:14:58.969892 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet a6454d815ae44cdda2fe8d013fca9b4b from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:14:58.970433 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet a6454d815ae44cdda2fe8d013fca9b4b from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-a6454d815ae44cdda2fe8d013fca9b4b
I20260709 13:14:58.974339 12260 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:58.975230 12260 tablet_bootstrap.cc:492] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:58.975979 12260 ts_tablet_manager.cc:1403] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 13:14:58.978919 12260 raft_consensus.cc:359] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.979698 12260 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:58.980325 12260 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:58.982823 12260 ts_tablet_manager.cc:1434] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:14:58.988620 11561 tablet_copy_source_session.cc:215] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:58.989926 11411 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-5e8ed42aaa39460bbea1f8deeccd0adc received from {username='slave'} at 127.0.0.1:50180
I20260709 13:14:58.990465 11411 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-5e8ed42aaa39460bbea1f8deeccd0adc on tablet 5e8ed42aaa39460bbea1f8deeccd0adc with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:58.996477 12257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6454d815ae44cdda2fe8d013fca9b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:59.012343 12257 tablet_copy_client.cc:806] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:59.012889 12257 tablet_copy_client.cc:670] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:59.016285 12257 tablet_copy_client.cc:538] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:59.024646 12257 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:14:59.044751 12257 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:59.046051 12257 tablet_bootstrap.cc:492] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:14:59.046681 12257 ts_tablet_manager.cc:1403] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:14:59.050551 12257 raft_consensus.cc:359] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:59.051520 12257 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:14:59.052140 12257 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:14:59.054702 12257 ts_tablet_manager.cc:1434] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.004s
I20260709 13:14:59.056676 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-a6454d815ae44cdda2fe8d013fca9b4b received from {username='slave'} at 127.0.0.1:41380
I20260709 13:14:59.057089 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-a6454d815ae44cdda2fe8d013fca9b4b on tablet a6454d815ae44cdda2fe8d013fca9b4b with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:14:59.173362 12123 raft_consensus.cc:1217] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:59.179831 12123 raft_consensus.cc:1217] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:59.367480 12123 raft_consensus.cc:1217] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:59.551321 12123 raft_consensus.cc:1217] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:59.629334 12248 raft_consensus.cc:1064] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:59.631229 12248 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:59.637692 12123 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.639694 11468 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.639153 12248 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:59.641223 11397 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.642575 12198 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:59.643596 12269 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:59.659489 12269 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.660631 11397 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.672127 12123 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.694329 11298 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.697036 11468 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.733153 11548 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:59.741324 12123 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:59.743476 11397 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:59.743139 12274 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:59.745139 12248 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:59.751241 12248 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.755563 11398 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.765798 12197 raft_consensus.cc:1064] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:59.768134 12197 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:59.772780 12123 raft_consensus.cc:2955] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.773916 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b150719d534f439aa33eb70af414a5d0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:59.793447 11300 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.812186 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet b150719d534f439aa33eb70af414a5d0 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:14:59.815773 12280 tablet_replica.cc:333] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:14:59.819727 12123 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.822451 11546 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.823838 12280 raft_consensus.cc:2243] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:59.825139 12256 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:59.825654 12280 raft_consensus.cc:2272] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:59.827775 12256 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:59.844256 12280 ts_tablet_manager.cc:1916] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:59.855254 12216 raft_consensus.cc:1064] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:14:59.857998 12216 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:59.884927 12123 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.886796 12208 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:14:59.889802 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64528.5us
I20260709 13:14:59.890655 11546 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.892133 12208 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:59.901624 11398 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.904351 11471 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:59.912588 12208 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:59.926793 12256 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:59.939239 12208 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.942070 11475 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.956631 12280 ts_tablet_manager.cc:1929] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:59.957171 12280 log.cc:1199] T b150719d534f439aa33eb70af414a5d0 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/b150719d534f439aa33eb70af414a5d0
I20260709 13:14:59.958765 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet b150719d534f439aa33eb70af414a5d0 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:14:59.964579 11300 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.970352 12123 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.984211 11548 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.984726 11399 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:14:59.988595 12281 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:14:59.992050 11544 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.000240 11399 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.002007 11475 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.003866 12286 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.018164 12286 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.019110 11475 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.022020 12122 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.028642 12216 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.031558 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5e8ed42aaa39460bbea1f8deeccd0adc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.028906 12122 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.048640 11297 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.056486 12123 raft_consensus.cc:2955] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.070421 11300 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:00.082324 12248 raft_consensus.cc:1064] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:00.082454 11471 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:00.101817 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e8ed42aaa39460bbea1f8deeccd0adc with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:00.108068 12294 tablet_replica.cc:333] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:00.109457 12294 raft_consensus.cc:2243] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:00.110749 12294 raft_consensus.cc:2272] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:00.112947 12248 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:00.117381 12122 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.119764 12256 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.126276 12294 ts_tablet_manager.cc:1916] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:00.126835 11399 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.132972 12256 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.137795 12197 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.141620 11399 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.150898 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dba44ee610b54495a5f1cfea0a876524 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.152791 12123 raft_consensus.cc:2955] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.157207 11398 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:00.157444 12122 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:00.158864 11477 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:00.159396 12269 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:00.161041 12198 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:00.164433 12248 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:00.183558 11300 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.213013 12294 ts_tablet_manager.cc:1929] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:00.214063 12294 log.cc:1199] T 5e8ed42aaa39460bbea1f8deeccd0adc P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/5e8ed42aaa39460bbea1f8deeccd0adc
I20260709 13:15:00.218358 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 5e8ed42aaa39460bbea1f8deeccd0adc (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:00.219278 12274 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.222705 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet dba44ee610b54495a5f1cfea0a876524 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:00.225848 12294 tablet_replica.cc:333] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:00.224488 12122 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.228726 11471 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.235687 12294 raft_consensus.cc:2243] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:00.236555 12294 raft_consensus.cc:2272] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:00.240386 12294 ts_tablet_manager.cc:1916] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:00.242579 11300 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.252985 11397 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.275027 12294 ts_tablet_manager.cc:1929] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:00.275647 12294 log.cc:1199] T dba44ee610b54495a5f1cfea0a876524 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/dba44ee610b54495a5f1cfea0a876524
I20260709 13:15:00.278817 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet dba44ee610b54495a5f1cfea0a876524 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:00.287323 11546 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:00.293511 11471 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.296267 12198 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.315246 12122 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:00.317173 12248 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.318782 12274 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:00.324398 11477 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.325587 12123 raft_consensus.cc:2955] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.334353 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a6454d815ae44cdda2fe8d013fca9b4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:00.339417 11298 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:00.357000 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet a6454d815ae44cdda2fe8d013fca9b4b with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:00.359171 12299 tablet_replica.cc:333] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:00.377434 12299 raft_consensus.cc:2243] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:00.378144 12299 raft_consensus.cc:2272] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:00.381748 12299 ts_tablet_manager.cc:1916] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:00.395288 12299 ts_tablet_manager.cc:1929] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:00.395700 12299 log.cc:1199] T a6454d815ae44cdda2fe8d013fca9b4b P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/a6454d815ae44cdda2fe8d013fca9b4b
I20260709 13:15:00.397084 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet a6454d815ae44cdda2fe8d013fca9b4b (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:01.449054 11546 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:15:01.456194 11515 consensus_peers.cc:597] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59af340b26264692af5490466dd69b78. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.456873 11398 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.456979 11475 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.458202 12300 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.458946 12248 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.467527 12300 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.469004 11398 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.476649 11475 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.482100 11398 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.486059 11300 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:15:01.495263 11366 consensus_peers.cc:597] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44539667cbc04657b801ff659dee5bde. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.496450 11546 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.497438 11475 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.497942 12301 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.498925 12208 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.514531 12216 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.520222 11544 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.522125 11475 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.533530 11546 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.540443 11297 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:01.559767 11515 consensus_peers.cc:597] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c4de37bdc334587a90c03b40bfd6c5f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.560349 11475 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.562072 11398 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.563755 12300 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.564711 12305 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.573975 12300 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.576498 11397 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.584384 11475 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.589148 11477 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.596553 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:15:01.600831 11441 consensus_peers.cc:597] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 902c85625c4647d481fa8ef07be336f4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.604952 11546 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.605250 11397 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:01.607345 12256 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.609185 12256 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:01.629733 12281 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.633601 11399 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.637444 11543 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.649747 11298 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:01.857432 12320 ts_tablet_manager.cc:933] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:01.864822 12320 tablet_copy_client.cc:323] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:15:01.866459 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 59af340b26264692af5490466dd69b78 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:15:01.866961 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 59af340b26264692af5490466dd69b78 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-59af340b26264692af5490466dd69b78
I20260709 13:15:01.873819 11561 tablet_copy_source_session.cc:215] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:01.877002 12320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af340b26264692af5490466dd69b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.883199 12322 ts_tablet_manager.cc:933] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:01.885119 12322 tablet_copy_client.cc:323] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:01.886876 11411 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 44539667cbc04657b801ff659dee5bde from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:01.887434 11411 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 44539667cbc04657b801ff659dee5bde from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-44539667cbc04657b801ff659dee5bde
I20260709 13:15:01.893096 12320 tablet_copy_client.cc:806] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:01.893671 12320 tablet_copy_client.cc:670] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:01.896343 11411 tablet_copy_source_session.cc:215] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:01.899176 12320 tablet_copy_client.cc:538] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.899914 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44539667cbc04657b801ff659dee5bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.910113 12320 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:01.913787 12322 tablet_copy_client.cc:806] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:01.914474 12322 tablet_copy_client.cc:670] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:01.919328 12322 tablet_copy_client.cc:538] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.929142 12322 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:01.935779 12320 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.938524 12320 tablet_bootstrap.cc:492] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:01.939316 12320 ts_tablet_manager.cc:1403] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.029s	user 0.024s	sys 0.001s
I20260709 13:15:01.942443 12320 raft_consensus.cc:359] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.944024 12320 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:01.944715 12320 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.955996 12320 ts_tablet_manager.cc:1434] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:15:01.961443 12322 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.962466 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-59af340b26264692af5490466dd69b78 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:15:01.962955 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-59af340b26264692af5490466dd69b78 on tablet 59af340b26264692af5490466dd69b78 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:01.962466 12322 tablet_bootstrap.cc:492] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:01.976050 12322 ts_tablet_manager.cc:1403] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.005s
I20260709 13:15:01.979235 12322 raft_consensus.cc:359] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.986536 12322 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:01.987303 12322 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:01.989758 12322 ts_tablet_manager.cc:1434] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:15:01.991447 11411 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-44539667cbc04657b801ff659dee5bde received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:01.991863 11411 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-44539667cbc04657b801ff659dee5bde on tablet 44539667cbc04657b801ff659dee5bde with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:02.145421 12322 ts_tablet_manager.cc:933] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:02.147285 12322 tablet_copy_client.cc:323] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:15:02.148983 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 3c4de37bdc334587a90c03b40bfd6c5f from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:15:02.149482 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 3c4de37bdc334587a90c03b40bfd6c5f from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-3c4de37bdc334587a90c03b40bfd6c5f
I20260709 13:15:02.155742 11561 tablet_copy_source_session.cc:215] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:02.158437 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4de37bdc334587a90c03b40bfd6c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:02.171099 12322 tablet_copy_client.cc:806] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:02.171756 12322 tablet_copy_client.cc:670] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:02.175405 12322 tablet_copy_client.cc:538] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:02.184494 12322 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:02.200902 12320 ts_tablet_manager.cc:933] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:02.201567 12322 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:02.202486 12322 tablet_bootstrap.cc:492] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:02.203176 12322 ts_tablet_manager.cc:1403] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.005s
I20260709 13:15:02.206205 12322 raft_consensus.cc:359] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:02.207059 12322 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:02.207692 12322 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:02.213481 12320 tablet_copy_client.cc:323] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:02.217190 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 902c85625c4647d481fa8ef07be336f4 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:02.217715 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 902c85625c4647d481fa8ef07be336f4 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-902c85625c4647d481fa8ef07be336f4
I20260709 13:15:02.219367 12322 ts_tablet_manager.cc:1434] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:15:02.228232 11487 tablet_copy_source_session.cc:215] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:02.231786 12320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 902c85625c4647d481fa8ef07be336f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:02.249497 12320 tablet_copy_client.cc:806] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:02.250195 12320 tablet_copy_client.cc:670] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:02.255481 12320 tablet_copy_client.cc:538] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:02.261518 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-3c4de37bdc334587a90c03b40bfd6c5f received from {username='slave'} at 127.0.0.1:41380
I20260709 13:15:02.262030 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-3c4de37bdc334587a90c03b40bfd6c5f on tablet 3c4de37bdc334587a90c03b40bfd6c5f with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:02.265203 12320 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:02.291985 12320 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:02.292734 12320 tablet_bootstrap.cc:492] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:02.293272 12320 ts_tablet_manager.cc:1403] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.000s
I20260709 13:15:02.295746 12320 raft_consensus.cc:359] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:02.296489 12320 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:02.297106 12320 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:02.299441 12320 ts_tablet_manager.cc:1434] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:15:02.301267 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-902c85625c4647d481fa8ef07be336f4 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:02.301669 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-902c85625c4647d481fa8ef07be336f4 on tablet 902c85625c4647d481fa8ef07be336f4 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:02.305534 12123 raft_consensus.cc:1217] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:02.486086 12123 raft_consensus.cc:1217] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:15:02.590544 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 53022.6us
I20260709 13:15:02.637765 12122 raft_consensus.cc:1217] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:02.668747 12122 raft_consensus.cc:1217] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:02.844585 12208 raft_consensus.cc:1064] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:02.846907 12208 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:02.860591 12122 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.862828 12333 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.872557 11477 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.874140 12208 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.874542 11543 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.876119 12335 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:02.904590 12333 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.906484 11477 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.911829 11546 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.926188 12122 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.931493 11300 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.957381 11399 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:02.966346 12122 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.967978 11477 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.970222 12333 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.971586 12208 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.979813 12208 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.982337 11475 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:02.986119 12122 raft_consensus.cc:2955] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.003155 11296 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.015599 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 44539667cbc04657b801ff659dee5bde with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:03.016817 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 44539667cbc04657b801ff659dee5bde with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:03.020982 12343 tablet_replica.cc:333] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:03.022079 12343 raft_consensus.cc:2243] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.022847 12343 raft_consensus.cc:2272] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.028582 12343 ts_tablet_manager.cc:1916] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:03.045558 12343 ts_tablet_manager.cc:1929] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:03.045970 12343 log.cc:1199] T 44539667cbc04657b801ff659dee5bde P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/44539667cbc04657b801ff659dee5bde
I20260709 13:15:03.047663 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 44539667cbc04657b801ff659dee5bde (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:03.052815 12305 raft_consensus.cc:1064] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:03.054946 12305 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.061482 12122 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.063107 11395 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.065105 12332 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.067277 12332 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:03.068405 11475 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.071651 12300 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.089444 12300 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.092170 11475 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.093420 12122 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.104516 11398 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.109656 11297 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:03.124889 11543 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.148424 12122 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.148769 11398 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.150678 12349 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.154897 12345 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:03.157282 12345 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.162352 11398 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.176028 12282 raft_consensus.cc:1064] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:03.179227 12282 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.188844 12332 raft_consensus.cc:1064] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:03.190197 11297 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.194335 12123 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.196200 12331 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:03.196311 12332 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.200181 11395 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.201216 11546 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.204242 12281 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.205797 12281 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.207576 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 59af340b26264692af5490466dd69b78 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:03.215444 12353 tablet_replica.cc:333] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:03.218837 12353 raft_consensus.cc:2243] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.215247 12122 raft_consensus.cc:2955] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.223217 12353 raft_consensus.cc:2272] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.232653 12353 ts_tablet_manager.cc:1916] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:03.239001 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 59af340b26264692af5490466dd69b78 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:03.249136 12351 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.252235 11543 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.258265 11395 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.264474 12353 ts_tablet_manager.cc:1929] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:03.264922 12353 log.cc:1199] T 59af340b26264692af5490466dd69b78 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/59af340b26264692af5490466dd69b78
I20260709 13:15:03.267091 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 59af340b26264692af5490466dd69b78 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:03.273725 12122 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.276139 12123 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.277622 12332 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.278929 11398 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.282212 11297 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:03.284258 12345 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.302302 11471 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:03.304288 11477 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.321008 12122 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.322336 12351 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.329523 12282 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.340051 12349 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.340696 12123 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.347328 12119 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.351732 12345 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.353703 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 902c85625c4647d481fa8ef07be336f4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:03.357414 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.359205 11543 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.366483 11297 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:03.376183 12351 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.380576 11398 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.391628 11471 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:15:03.421072 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45090us
I20260709 13:15:03.428344 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 902c85625c4647d481fa8ef07be336f4 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:03.436057 11546 raft_consensus.cc:2955] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.456745 12362 tablet_replica.cc:333] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:03.457898 12362 raft_consensus.cc:2243] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.458719 12362 raft_consensus.cc:2272] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.460909 11543 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:03.462581 12362 ts_tablet_manager.cc:1916] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:03.466449 12119 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.470603 12349 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.471339 11395 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:03.476692 12349 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.484620 12349 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.490747 12119 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.491504 11395 raft_consensus.cc:2955] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.512019 11296 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:03.514344 12362 ts_tablet_manager.cc:1929] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:03.512045 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3c4de37bdc334587a90c03b40bfd6c5f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:03.514894 12362 log.cc:1199] T 902c85625c4647d481fa8ef07be336f4 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/902c85625c4647d481fa8ef07be336f4
I20260709 13:15:03.521426 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 902c85625c4647d481fa8ef07be336f4 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:03.525426 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c4de37bdc334587a90c03b40bfd6c5f with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:03.526463 12353 tablet_replica.cc:333] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:03.527516 12353 raft_consensus.cc:2243] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.528168 12353 raft_consensus.cc:2272] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.531508 12353 ts_tablet_manager.cc:1916] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:03.543447 12353 ts_tablet_manager.cc:1929] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:03.543805 12353 log.cc:1199] T 3c4de37bdc334587a90c03b40bfd6c5f P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/3c4de37bdc334587a90c03b40bfd6c5f
I20260709 13:15:03.545122 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 3c4de37bdc334587a90c03b40bfd6c5f (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:04.594520 11477 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:15:04.601192 11441 consensus_peers.cc:597] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 058fa5ec3edb42c796103c7361c1a8e1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.601861 11543 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:04.603418 12351 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.603960 11398 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:04.605935 12351 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.613785 12331 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.617079 11398 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.625023 11543 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.628358 11298 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.629021 11398 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:04.636312 11546 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:04.639604 11475 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:04.639971 11366 consensus_peers.cc:597] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79d2059876e34ff89bf967e96ab7bf09. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.640563 12208 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.641911 12335 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:04.661799 12335 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.669945 11548 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.674742 11471 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.682832 11398 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:04.701342 11471 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:04.701968 11366 consensus_peers.cc:597] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f615bf163b9483d8209eb6c8fedca15. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.702521 11297 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.704672 12208 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.718636 11548 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:04.720207 12376 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:04.722862 12208 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.727301 11471 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.739888 11398 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:04.743463 11297 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.750593 11548 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.778825 11543 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:04.778903 11366 consensus_peers.cc:597] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d02abb4f52449679a2415346f354340. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.780669 12208 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.781401 11471 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:04.783509 12376 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.797824 12376 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.799288 11475 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.799615 11548 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:04.814721 11298 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:05.053416 12383 ts_tablet_manager.cc:933] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:05.056775 12383 tablet_copy_client.cc:323] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:05.059134 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 058fa5ec3edb42c796103c7361c1a8e1 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:05.059597 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 058fa5ec3edb42c796103c7361c1a8e1 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-058fa5ec3edb42c796103c7361c1a8e1
I20260709 13:15:05.066922 11487 tablet_copy_source_session.cc:215] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:05.070111 12383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058fa5ec3edb42c796103c7361c1a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:05.085525 12383 tablet_copy_client.cc:806] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:05.086249 12383 tablet_copy_client.cc:670] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:05.089998 12383 tablet_copy_client.cc:538] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:05.098968 12383 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:05.119239 12383 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:05.120154 12383 tablet_bootstrap.cc:492] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:05.121583 12383 ts_tablet_manager.cc:1403] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 13:15:05.124080 12386 ts_tablet_manager.cc:933] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:05.124818 12383 raft_consensus.cc:359] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.127298 12383 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:05.128013 12383 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.131348 12383 ts_tablet_manager.cc:1434] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:15:05.134920 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-058fa5ec3edb42c796103c7361c1a8e1 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:05.135336 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-058fa5ec3edb42c796103c7361c1a8e1 on tablet 058fa5ec3edb42c796103c7361c1a8e1 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:05.144899 12386 tablet_copy_client.cc:323] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:05.146386 11411 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 79d2059876e34ff89bf967e96ab7bf09 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:05.146839 11411 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 79d2059876e34ff89bf967e96ab7bf09 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-79d2059876e34ff89bf967e96ab7bf09
I20260709 13:15:05.155141 11411 tablet_copy_source_session.cc:215] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:05.157800 12386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d2059876e34ff89bf967e96ab7bf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:05.170920 12386 tablet_copy_client.cc:806] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:05.171552 12386 tablet_copy_client.cc:670] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:05.175681 12386 tablet_copy_client.cc:538] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:05.182687 12386 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:05.200510 12386 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:05.201279 12386 tablet_bootstrap.cc:492] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:05.201917 12386 ts_tablet_manager.cc:1403] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.005s
I20260709 13:15:05.204421 12386 raft_consensus.cc:359] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.205065 12386 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:05.206089 12386 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.208292 12386 ts_tablet_manager.cc:1434] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:15:05.209997 11411 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-79d2059876e34ff89bf967e96ab7bf09 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:05.210393 11411 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-79d2059876e34ff89bf967e96ab7bf09 on tablet 79d2059876e34ff89bf967e96ab7bf09 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:05.310717 12386 ts_tablet_manager.cc:933] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:05.340685 12386 tablet_copy_client.cc:323] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:05.346570 11411 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 4f615bf163b9483d8209eb6c8fedca15 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:05.347136 11411 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 4f615bf163b9483d8209eb6c8fedca15 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-4f615bf163b9483d8209eb6c8fedca15
I20260709 13:15:05.354148 12383 ts_tablet_manager.cc:933] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:05.356331 12383 tablet_copy_client.cc:323] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:05.359429 11411 tablet_copy_source_session.cc:215] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:05.359494 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 6d02abb4f52449679a2415346f354340 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:05.360242 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 6d02abb4f52449679a2415346f354340 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-6d02abb4f52449679a2415346f354340
I20260709 13:15:05.365903 12386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f615bf163b9483d8209eb6c8fedca15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:05.371832 11412 tablet_copy_source_session.cc:215] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:05.375062 12383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d02abb4f52449679a2415346f354340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:05.383673 12386 tablet_copy_client.cc:806] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:05.386070 12386 tablet_copy_client.cc:670] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:05.394553 12386 tablet_copy_client.cc:538] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:05.400528 12383 tablet_copy_client.cc:806] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:05.401160 12383 tablet_copy_client.cc:670] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:05.405519 12386 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:05.406654 12383 tablet_copy_client.cc:538] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:05.420791 12383 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:05.428491 12386 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:05.429410 12386 tablet_bootstrap.cc:492] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:05.430099 12386 ts_tablet_manager.cc:1403] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.008s
I20260709 13:15:05.433142 12386 raft_consensus.cc:359] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.433990 12386 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:05.434671 12386 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.437696 12386 ts_tablet_manager.cc:1434] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:05.440686 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-4f615bf163b9483d8209eb6c8fedca15 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:05.441097 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-4f615bf163b9483d8209eb6c8fedca15 on tablet 4f615bf163b9483d8209eb6c8fedca15 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:05.447315 12383 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:05.448243 12383 tablet_bootstrap.cc:492] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:05.448889 12383 ts_tablet_manager.cc:1403] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.009s
I20260709 13:15:05.451817 12383 raft_consensus.cc:359] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.452594 12383 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:05.453207 12383 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:05.455534 12383 ts_tablet_manager.cc:1434] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:15:05.457103 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6d02abb4f52449679a2415346f354340 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:05.457499 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6d02abb4f52449679a2415346f354340 on tablet 6d02abb4f52449679a2415346f354340 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:05.579144 12123 raft_consensus.cc:1217] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:05.610335 12123 raft_consensus.cc:1217] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:05.908630 12123 raft_consensus.cc:1217] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:05.972785 12123 raft_consensus.cc:1217] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:06.131621 12397 raft_consensus.cc:1064] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:06.134111 12397 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.176192 12391 raft_consensus.cc:1064] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:06.178622 12391 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.193532 12123 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.197260 11543 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.197829 12208 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:06.199261 12380 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:06.217988 11475 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.221365 12401 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:06.228451 11548 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.232460 11399 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.237496 12400 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:06.240870 12400 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:15:06.244396 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 106823us
I20260709 13:15:06.246352 12119 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.251036 12390 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:06.303949 12403 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.339337 12123 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.411669 12401 raft_consensus.cc:1064] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:06.430706 12380 raft_consensus.cc:1064] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
W20260709 13:15:06.433506 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 197587us
I20260709 13:15:06.417047 12401 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.401319 12399 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.500021 11399 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.507975 11296 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.434814 12380 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.623085 11548 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.685524 12119 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.685354 12123 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.687605 11477 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.686825 11468 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.691139 11546 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.700911 12401 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:06.713361 11471 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:15:06.728523 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62791.7us
I20260709 13:15:06.723379 12401 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:15:06.734462 12122 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.741118 12391 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:06.752305 11543 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.753214 11545 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:06.761189 12426 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:06.781078 11298 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.795846 12403 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:06.804723 12429 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:15:06.869644 12391 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.871160 11550 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.876407 12403 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.883332 11477 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.885681 11546 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.896944 12122 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.919813 11477 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.928068 12122 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.946612 11398 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.956571 12122 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:06.958060 11543 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:06.960237 12391 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:06.961542 12429 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:06.968700 12429 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.972520 12118 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.973891 11550 raft_consensus.cc:2955] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:06.974576 11477 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:06.993417 11546 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:06.982973 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 79d2059876e34ff89bf967e96ab7bf09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.989192 12122 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:06.995462 12399 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:06.982573 11296 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.996258 12400 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:06.997167 11296 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:07.002964 12400 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.005596 12122 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.013355 11546 raft_consensus.cc:2955] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.020788 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 058fa5ec3edb42c796103c7361c1a8e1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:07.034865 11297 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.038833 11297 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.060518 11397 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:07.060668 11398 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:07.067471 11477 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:07.068352 12122 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:07.070107 12424 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:07.071933 12424 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:07.072757 11546 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:07.078500 12429 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:07.079672 12403 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.081373 12122 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.090934 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 058fa5ec3edb42c796103c7361c1a8e1 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:07.092773 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 79d2059876e34ff89bf967e96ab7bf09 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:07.092526 12429 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.094321 11546 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.100871 11473 raft_consensus.cc:2955] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.104779 12442 tablet_replica.cc:333] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:07.105870 12441 tablet_replica.cc:333] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:07.113318 12118 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:07.115123 12425 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:07.119975 11300 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.124192 12442 raft_consensus.cc:2243] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.127877 12442 raft_consensus.cc:2272] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.132088 12442 ts_tablet_manager.cc:1916] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:07.133247 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4f615bf163b9483d8209eb6c8fedca15 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:07.134128 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d02abb4f52449679a2415346f354340 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:07.122097 12118 raft_consensus.cc:2955] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.142908 12441 raft_consensus.cc:2243] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.143743 12441 raft_consensus.cc:2272] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.147799 12441 ts_tablet_manager.cc:1916] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:07.152550 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f615bf163b9483d8209eb6c8fedca15 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:07.153287 11299 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:07.154558 12443 tablet_replica.cc:333] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:07.155745 12443 raft_consensus.cc:2243] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.156553 12443 raft_consensus.cc:2272] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.157521 12442 ts_tablet_manager.cc:1929] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:07.158103 12442 log.cc:1199] T 79d2059876e34ff89bf967e96ab7bf09 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/79d2059876e34ff89bf967e96ab7bf09
I20260709 13:15:07.160959 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 79d2059876e34ff89bf967e96ab7bf09 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:07.166891 12443 ts_tablet_manager.cc:1916] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:07.175278 12441 ts_tablet_manager.cc:1929] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:07.175865 12441 log.cc:1199] T 058fa5ec3edb42c796103c7361c1a8e1 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/058fa5ec3edb42c796103c7361c1a8e1
I20260709 13:15:07.178761 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 058fa5ec3edb42c796103c7361c1a8e1 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:07.183959 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d02abb4f52449679a2415346f354340 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:07.184904 12442 tablet_replica.cc:333] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:07.185489 12443 ts_tablet_manager.cc:1929] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:07.186030 12442 raft_consensus.cc:2243] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:07.186957 12442 raft_consensus.cc:2272] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:07.186039 12443 log.cc:1199] T 4f615bf163b9483d8209eb6c8fedca15 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/4f615bf163b9483d8209eb6c8fedca15
I20260709 13:15:07.193087 12442 ts_tablet_manager.cc:1916] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:07.194002 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 4f615bf163b9483d8209eb6c8fedca15 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:07.204903 12442 ts_tablet_manager.cc:1929] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:07.205369 12442 log.cc:1199] T 6d02abb4f52449679a2415346f354340 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/6d02abb4f52449679a2415346f354340
I20260709 13:15:07.208271 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 6d02abb4f52449679a2415346f354340 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:08.251200 11477 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.257853 11550 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:08.259430 12400 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:08.262133 11441 consensus_peers.cc:597] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e106bf87136140b4bebad713d83d02fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.265486 11397 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:08.266782 12400 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:08.273360 12400 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.278849 11546 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.280742 11397 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.295053 11477 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.305114 11399 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:08.306221 11550 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:08.307252 11441 consensus_peers.cc:597] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56b606c6942f42cf914bcabc7e2bc502. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.316001 11298 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.322201 12450 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:08.323479 12445 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:08.332451 12449 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.344451 11397 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.344344 11550 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.364364 11397 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.379963 11298 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:15:08.394588 11366 consensus_peers.cc:597] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b66bfe782a6d489fa35dc079b8c4d0a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.397756 11477 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:08.399566 12444 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:08.402801 11550 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:08.404320 12429 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:08.409685 12444 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.411118 11477 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.429523 11299 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.432567 11397 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.443818 11550 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:15:08.459384 11366 consensus_peers.cc:597] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6eba703fc47843af924d4e45f12db16b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:08.460752 11543 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:08.461633 11477 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:08.463024 12460 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:08.465723 12460 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:08.476495 12429 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.482770 11550 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.483832 11477 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.502420 11300 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 3, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:08.651165 12467 ts_tablet_manager.cc:933] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:08.653201 12467 tablet_copy_client.cc:323] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:08.655201 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet e106bf87136140b4bebad713d83d02fe from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:08.655781 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet e106bf87136140b4bebad713d83d02fe from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-e106bf87136140b4bebad713d83d02fe
I20260709 13:15:08.666704 11487 tablet_copy_source_session.cc:215] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.675105 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e106bf87136140b4bebad713d83d02fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.692353 12467 tablet_copy_client.cc:806] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.692970 12467 tablet_copy_client.cc:670] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.696749 12467 tablet_copy_client.cc:538] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.705565 12467 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:08.733104 12467 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:08.733968 12467 tablet_bootstrap.cc:492] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:08.734612 12467 ts_tablet_manager.cc:1403] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.008s
I20260709 13:15:08.737010 12467 raft_consensus.cc:359] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.737620 12467 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:08.738188 12467 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.741772 12467 ts_tablet_manager.cc:1434] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:15:08.743607 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-e106bf87136140b4bebad713d83d02fe received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:08.744064 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-e106bf87136140b4bebad713d83d02fe on tablet e106bf87136140b4bebad713d83d02fe with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:08.792881 12467 ts_tablet_manager.cc:933] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:08.795287 12467 tablet_copy_client.cc:323] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:08.796921 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 56b606c6942f42cf914bcabc7e2bc502 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:08.797448 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 56b606c6942f42cf914bcabc7e2bc502 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-56b606c6942f42cf914bcabc7e2bc502
I20260709 13:15:08.804775 11487 tablet_copy_source_session.cc:215] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.808033 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b606c6942f42cf914bcabc7e2bc502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.824913 12467 tablet_copy_client.cc:806] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.825448 12467 tablet_copy_client.cc:670] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.828625 12467 tablet_copy_client.cc:538] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.836720 12467 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:08.857185 12467 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:08.857306 12473 ts_tablet_manager.cc:933] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:08.857965 12467 tablet_bootstrap.cc:492] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:08.858608 12467 ts_tablet_manager.cc:1403] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:15:08.859421 12473 tablet_copy_client.cc:323] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:08.861624 12467 raft_consensus.cc:359] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.862543 12467 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:08.862747 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 6eba703fc47843af924d4e45f12db16b from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:08.863346 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 6eba703fc47843af924d4e45f12db16b from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-6eba703fc47843af924d4e45f12db16b
I20260709 13:15:08.863287 12467 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:08.869092 12467 ts_tablet_manager.cc:1434] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 13:15:08.873942 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-56b606c6942f42cf914bcabc7e2bc502 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:08.874389 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-56b606c6942f42cf914bcabc7e2bc502 on tablet 56b606c6942f42cf914bcabc7e2bc502 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:08.884333 12467 ts_tablet_manager.cc:933] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:08.887040 11412 tablet_copy_source_session.cc:215] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.888450 12467 tablet_copy_client.cc:323] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:08.890528 12473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba703fc47843af924d4e45f12db16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.890892 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet b66bfe782a6d489fa35dc079b8c4d0a5 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:08.891784 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet b66bfe782a6d489fa35dc079b8c4d0a5 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-b66bfe782a6d489fa35dc079b8c4d0a5
I20260709 13:15:08.898177 11412 tablet_copy_source_session.cc:215] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.901062 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b66bfe782a6d489fa35dc079b8c4d0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.908041 12473 tablet_copy_client.cc:806] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.908665 12473 tablet_copy_client.cc:670] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.912607 12473 tablet_copy_client.cc:538] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.915127 12467 tablet_copy_client.cc:806] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.915685 12467 tablet_copy_client.cc:670] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.918769 12467 tablet_copy_client.cc:538] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.933998 12473 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:08.934556 12467 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:09.086805 12467 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:09.087809 12467 tablet_bootstrap.cc:492] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:09.088510 12467 ts_tablet_manager.cc:1403] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.154s	user 0.016s	sys 0.012s
I20260709 13:15:09.088642 12473 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:09.089555 12473 tablet_bootstrap.cc:492] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:09.090188 12473 ts_tablet_manager.cc:1403] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.156s	user 0.020s	sys 0.009s
I20260709 13:15:09.093106 12473 raft_consensus.cc:359] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:09.093928 12473 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:09.094570 12467 raft_consensus.cc:359] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:09.095490 12467 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:09.096186 12467 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:09.121939 12467 ts_tablet_manager.cc:1434] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:15:09.094535 12473 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:09.124544 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b66bfe782a6d489fa35dc079b8c4d0a5 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:09.124998 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b66bfe782a6d489fa35dc079b8c4d0a5 on tablet b66bfe782a6d489fa35dc079b8c4d0a5 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:09.125728 12473 ts_tablet_manager.cc:1434] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:15:09.127844 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6eba703fc47843af924d4e45f12db16b received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:09.128216 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6eba703fc47843af924d4e45f12db16b on tablet 6eba703fc47843af924d4e45f12db16b with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:09.133495 12118 raft_consensus.cc:1217] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:09.337390 12122 raft_consensus.cc:1217] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:09.373764 12122 raft_consensus.cc:1217] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:09.475150 12122 raft_consensus.cc:1217] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:15:09.523554 12482 raft_consensus.cc:1064] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:09.526569 12482 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.534519 12122 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.536736 11543 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.536800 12481 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:09.538571 11395 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.540052 12445 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:09.543574 12445 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:09.555519 12449 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.566331 11395 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.575716 11543 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.577519 12122 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.584581 11298 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:09.622617 11477 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.630324 12122 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:09.631592 11395 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:09.632854 12466 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:09.633749 12445 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:09.642077 12445 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.645856 12122 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.646098 11398 raft_consensus.cc:2955] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.670197 11297 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.677675 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e106bf87136140b4bebad713d83d02fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:09.722780 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet e106bf87136140b4bebad713d83d02fe with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:09.726209 12491 tablet_replica.cc:333] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:09.728235 12491 raft_consensus.cc:2243] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:09.728955 12491 raft_consensus.cc:2272] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:09.742640 12491 ts_tablet_manager.cc:1916] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:09.755743 12491 ts_tablet_manager.cc:1929] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:09.756103 12491 log.cc:1199] T e106bf87136140b4bebad713d83d02fe P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/e106bf87136140b4bebad713d83d02fe
I20260709 13:15:09.757688 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet e106bf87136140b4bebad713d83d02fe (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:09.821091 12479 raft_consensus.cc:1064] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:09.823438 12479 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.837008 12122 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.837723 11550 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.838575 12460 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:09.839504 12493 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:09.843196 11477 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:09.847334 12460 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:09.855618 12429 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.857322 11477 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.873992 12122 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.876327 11297 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:09.890759 12497 raft_consensus.cc:1064] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:09.900497 11543 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.913664 12497 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.921888 11546 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:09.923318 12497 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:09.924081 11477 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:09.925015 12122 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:09.925976 12498 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:09.928768 12429 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:09.934782 11395 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.944767 12118 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:09.946971 12466 raft_consensus.cc:1064] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:09.949087 11543 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:09.949589 12466 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:09.950703 12429 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:09.963275 12498 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.964242 12499 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.966893 12479 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:09.977587 12118 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.979624 11550 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.975184 11543 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.987226 11477 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.993870 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b66bfe782a6d489fa35dc079b8c4d0a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:10.004382 11299 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:09.973467 12122 raft_consensus.cc:2955] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.017660 11543 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:10.018893 11398 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:10.019264 12118 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:10.019301 12466 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:10.028762 12445 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:10.031981 12480 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:10.039193 11398 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:10.047961 11300 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.061311 12122 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:10.061847 11477 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:10.064328 12466 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.066164 12499 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:10.068092 12499 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:10.074056 11395 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.076581 12500 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.080695 12122 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.079448 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet b66bfe782a6d489fa35dc079b8c4d0a5 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:10.088577 12118 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.079092 11477 raft_consensus.cc:2955] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.100453 12509 tablet_replica.cc:333] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:10.101827 12509 raft_consensus.cc:2243] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:10.102753 12509 raft_consensus.cc:2272] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.102763 11550 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.106541 12509 ts_tablet_manager.cc:1916] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:10.107800 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6eba703fc47843af924d4e45f12db16b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:10.114073 11299 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 4 to 5, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.117632 11300 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:10.136866 12509 ts_tablet_manager.cc:1929] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:10.137035 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eba703fc47843af924d4e45f12db16b with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:10.137499 12509 log.cc:1199] T b66bfe782a6d489fa35dc079b8c4d0a5 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/b66bfe782a6d489fa35dc079b8c4d0a5
I20260709 13:15:10.138012 12491 tablet_replica.cc:333] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:10.139063 12491 raft_consensus.cc:2243] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:10.139595 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet b66bfe782a6d489fa35dc079b8c4d0a5 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:10.139890 12491 raft_consensus.cc:2272] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.143739 12491 ts_tablet_manager.cc:1916] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:10.145732 11477 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:10.152582 12118 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:10.155602 12445 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:10.161731 11550 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:10.164119 12466 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:10.166016 12466 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.167630 12118 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.183063 11543 raft_consensus.cc:2955] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.197813 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 56b606c6942f42cf914bcabc7e2bc502 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:10.200415 12491 ts_tablet_manager.cc:1929] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:15:10.200946 12491 log.cc:1199] T 6eba703fc47843af924d4e45f12db16b P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/6eba703fc47843af924d4e45f12db16b
I20260709 13:15:10.202526 11299 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:10.206725 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 6eba703fc47843af924d4e45f12db16b (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:10.216833 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 56b606c6942f42cf914bcabc7e2bc502 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:10.222728 12510 tablet_replica.cc:333] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:10.223764 12510 raft_consensus.cc:2243] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:10.224596 12510 raft_consensus.cc:2272] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.228005 12510 ts_tablet_manager.cc:1916] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:10.239159 12510 ts_tablet_manager.cc:1929] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:10.239454 12510 log.cc:1199] T 56b606c6942f42cf914bcabc7e2bc502 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/56b606c6942f42cf914bcabc7e2bc502
I20260709 13:15:10.240944 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 56b606c6942f42cf914bcabc7e2bc502 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:11.285856 11550 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.292347 11477 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.292680 11398 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:11.293037 11515 consensus_peers.cc:597] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43c1c5858a0f4571a21e2d097910aa44. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:11.293942 12422 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.294747 12513 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.302356 12513 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.304186 11398 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.306126 11477 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.322038 11297 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.333310 11477 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.341658 11550 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.342765 11399 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:11.343109 11441 consensus_peers.cc:597] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e9e663b46c5417b9a0e42fdc7b70969. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:11.344551 12466 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.346357 12466 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.389358 12512 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.399523 11550 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.402644 11399 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.406687 11298 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:11.425719 11399 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:15:11.434654 11366 consensus_peers.cc:597] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e90063696c0437bb8ad2d545f45a9ce. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:11.435599 11477 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.435495 11543 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.437063 12514 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.438072 12500 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.450899 12514 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.452715 11543 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.465361 11468 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.475632 11477 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.481911 11543 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.483703 12466 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:11.485663 11441 consensus_peers.cc:597] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b10b16798ae24794b186d5daa625c6d1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:11.486539 11398 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:11.476619 11297 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:11.489233 12466 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:11.507365 12482 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.510888 11543 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.515722 11399 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.534950 11297 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:11.765719 12536 ts_tablet_manager.cc:933] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:11.767560 12536 tablet_copy_client.cc:323] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:11.769078 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 5e9e663b46c5417b9a0e42fdc7b70969 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:11.769631 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 5e9e663b46c5417b9a0e42fdc7b70969 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-5e9e663b46c5417b9a0e42fdc7b70969
I20260709 13:15:11.778090 11487 tablet_copy_source_session.cc:215] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:11.781095 12536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e9e663b46c5417b9a0e42fdc7b70969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:11.796571 12536 tablet_copy_client.cc:806] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:11.797353 12536 tablet_copy_client.cc:670] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:11.798768 12538 ts_tablet_manager.cc:933] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:11.802063 12538 tablet_copy_client.cc:323] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:15:11.803996 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 43c1c5858a0f4571a21e2d097910aa44 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:15:11.804523 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 43c1c5858a0f4571a21e2d097910aa44 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-43c1c5858a0f4571a21e2d097910aa44
I20260709 13:15:11.804697 12536 tablet_copy_client.cc:538] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:11.812222 11561 tablet_copy_source_session.cc:215] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:11.814479 12536 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:11.816404 12538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43c1c5858a0f4571a21e2d097910aa44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:11.830046 12538 tablet_copy_client.cc:806] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:11.830663 12538 tablet_copy_client.cc:670] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:11.835626 12538 tablet_copy_client.cc:538] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:11.847121 12538 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:11.887372 12538 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:11.888350 12538 tablet_bootstrap.cc:492] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:11.889127 12538 ts_tablet_manager.cc:1403] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.014s
I20260709 13:15:11.892993 12538 raft_consensus.cc:359] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.895491 12538 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:11.896234 12538 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.926172 12538 ts_tablet_manager.cc:1434] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.007s
I20260709 13:15:11.936183 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-43c1c5858a0f4571a21e2d097910aa44 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:15:11.936661 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-43c1c5858a0f4571a21e2d097910aa44 on tablet 43c1c5858a0f4571a21e2d097910aa44 with peer b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:11.944877 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71199.5us
I20260709 13:15:11.963356 12536 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:11.964282 12536 tablet_bootstrap.cc:492] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:11.964982 12536 ts_tablet_manager.cc:1403] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.151s	user 0.044s	sys 0.068s
I20260709 13:15:11.967844 12536 raft_consensus.cc:359] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.968717 12536 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:11.969357 12536 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:11.971834 12536 ts_tablet_manager.cc:1434] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:15:11.975301 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-5e9e663b46c5417b9a0e42fdc7b70969 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:11.975729 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-5e9e663b46c5417b9a0e42fdc7b70969 on tablet 5e9e663b46c5417b9a0e42fdc7b70969 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:12.018314 12536 ts_tablet_manager.cc:933] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:12.019961 12536 tablet_copy_client.cc:323] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:12.021284 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 8e90063696c0437bb8ad2d545f45a9ce from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:12.021785 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 8e90063696c0437bb8ad2d545f45a9ce from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-8e90063696c0437bb8ad2d545f45a9ce
I20260709 13:15:12.026566 11412 tablet_copy_source_session.cc:215] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:12.029422 12536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e90063696c0437bb8ad2d545f45a9ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.048234 12536 tablet_copy_client.cc:806] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:12.048857 12536 tablet_copy_client.cc:670] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:12.049932 12538 ts_tablet_manager.cc:933] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:12.051568 12538 tablet_copy_client.cc:323] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:15:12.052383 12536 tablet_copy_client.cc:538] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:12.053059 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet b10b16798ae24794b186d5daa625c6d1 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:15:12.053525 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet b10b16798ae24794b186d5daa625c6d1 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-b10b16798ae24794b186d5daa625c6d1
I20260709 13:15:12.060011 11487 tablet_copy_source_session.cc:215] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:12.062148 12536 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:12.062839 12538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10b16798ae24794b186d5daa625c6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.080071 12538 tablet_copy_client.cc:806] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:12.080863 12538 tablet_copy_client.cc:670] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:12.085295 12538 tablet_copy_client.cc:538] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:12.092379 12536 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:12.093333 12536 tablet_bootstrap.cc:492] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:12.093998 12536 ts_tablet_manager.cc:1403] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.009s
I20260709 13:15:12.094347 12538 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:12.097050 12536 raft_consensus.cc:359] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:12.097920 12536 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:12.098661 12536 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:12.101352 12536 ts_tablet_manager.cc:1434] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
I20260709 13:15:12.103178 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-8e90063696c0437bb8ad2d545f45a9ce received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:12.103618 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-8e90063696c0437bb8ad2d545f45a9ce on tablet 8e90063696c0437bb8ad2d545f45a9ce with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:12.127192 12538 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:12.128194 12538 tablet_bootstrap.cc:492] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:12.128885 12538 ts_tablet_manager.cc:1403] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.004s
I20260709 13:15:12.132019 12538 raft_consensus.cc:359] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:12.132813 12538 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:12.133422 12538 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:12.135844 12538 ts_tablet_manager.cc:1434] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:12.137411 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b10b16798ae24794b186d5daa625c6d1 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:15:12.137784 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b10b16798ae24794b186d5daa625c6d1 on tablet b10b16798ae24794b186d5daa625c6d1 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:12.291575 12122 raft_consensus.cc:1217] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:12.395459 12122 raft_consensus.cc:1217] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:12.402029 12122 raft_consensus.cc:1217] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:12.603384 12122 raft_consensus.cc:1217] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:12.873526 12466 raft_consensus.cc:1064] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:12.875355 12466 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:12.880847 12122 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:12.881507 11543 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:12.882666 11399 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:12.884167 12482 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:12.886368 12535 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:12.887158 12551 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:12.916322 12535 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:12.927959 12122 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:12.928921 11397 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:12.935382 11546 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:12.944340 12519 raft_consensus.cc:1064] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:12.948930 12519 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:12.952666 11299 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:12.964263 12549 raft_consensus.cc:1064] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:12.977440 12549 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.000722 11477 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.001300 11397 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.002200 12122 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.003208 12516 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.009104 12543 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.011099 12543 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:13.040410 12519 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.053522 11468 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.070557 11397 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.073627 12122 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.075348 12556 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.084260 12118 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.099912 11298 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.108994 11543 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.110389 12556 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.106666 11468 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.119920 11477 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.123463 12557 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:13.123361 11543 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.130057 12119 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.131533 12561 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.132000 11475 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.133760 12561 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.142271 12561 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.147311 12122 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.148727 12555 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.150312 11473 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.155926 12535 raft_consensus.cc:1064] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:13.158483 12535 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.163079 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 43c1c5858a0f4571a21e2d097910aa44 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:13.164405 12556 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.170466 12123 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.152854 12122 raft_consensus.cc:2955] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.182126 11298 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.199929 11543 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.202457 12555 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:13.205057 12482 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.205533 12118 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.212152 11397 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.214335 12551 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:13.215333 12466 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.205469 12122 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.228893 11546 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:13.232631 12551 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:13.225780 11300 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.242802 11546 raft_consensus.cc:2955] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.244889 11550 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.254387 11477 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.264989 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 43c1c5858a0f4571a21e2d097910aa44 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:13.270561 12575 tablet_replica.cc:333] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:13.270445 11399 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.273017 12575 raft_consensus.cc:2243] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:13.272099 12551 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.275488 11395 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.277736 11300 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 3 to 4, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.278736 12575 raft_consensus.cc:2272] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:13.286257 12575 ts_tablet_manager.cc:1916] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:13.297992 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5e9e663b46c5417b9a0e42fdc7b70969 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:13.304011 12122 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.310881 12575 ts_tablet_manager.cc:1929] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:13.311265 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e9e663b46c5417b9a0e42fdc7b70969 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:13.311518 12575 log.cc:1199] T 43c1c5858a0f4571a21e2d097910aa44 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/43c1c5858a0f4571a21e2d097910aa44
I20260709 13:15:13.321846 11546 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.330514 11544 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.332008 12549 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.333446 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 43c1c5858a0f4571a21e2d097910aa44 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:13.333662 12575 tablet_replica.cc:333] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:15:13.336952 12122 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.339124 12575 raft_consensus.cc:2243] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:13.339805 12575 raft_consensus.cc:2272] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:13.338758 12499 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.342372 11300 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.340023 12556 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:13.346966 12122 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.356545 12575 ts_tablet_manager.cc:1916] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:13.368546 11546 raft_consensus.cc:2955] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.378549 11298 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 3 to 4, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.379297 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e90063696c0437bb8ad2d545f45a9ce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:13.395740 12575 ts_tablet_manager.cc:1929] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:13.396317 12575 log.cc:1199] T 5e9e663b46c5417b9a0e42fdc7b70969 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/5e9e663b46c5417b9a0e42fdc7b70969
I20260709 13:15:13.397397 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e90063696c0437bb8ad2d545f45a9ce with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43626
I20260709 13:15:13.402693 12577 tablet_replica.cc:333] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:15:13.403438 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 5e9e663b46c5417b9a0e42fdc7b70969 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:13.408679 12577 raft_consensus.cc:2243] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:13.409499 12577 raft_consensus.cc:2272] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:13.413389 12577 ts_tablet_manager.cc:1916] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:13.417450 11477 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:13.429778 12122 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.431440 11395 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:13.431389 12551 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.438482 12551 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:13.444099 12577 ts_tablet_manager.cc:1929] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:13.444612 12577 log.cc:1199] T 8e90063696c0437bb8ad2d545f45a9ce P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/8e90063696c0437bb8ad2d545f45a9ce
I20260709 13:15:13.446130 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 8e90063696c0437bb8ad2d545f45a9ce (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:13.448210 12555 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.457723 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b10b16798ae24794b186d5daa625c6d1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:13.457635 12122 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.458937 11395 raft_consensus.cc:2955] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:13.472589 11297 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 3 to 4, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 1 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.485107 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet b10b16798ae24794b186d5daa625c6d1 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43418
I20260709 13:15:13.496527 12578 tablet_replica.cc:333] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:15:13.497303 12578 raft_consensus.cc:2243] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:13.497958 12578 raft_consensus.cc:2272] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:13.501063 12578 ts_tablet_manager.cc:1916] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:13.513890 12578 ts_tablet_manager.cc:1929] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:13.514248 12578 log.cc:1199] T b10b16798ae24794b186d5daa625c6d1 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/b10b16798ae24794b186d5daa625c6d1
I20260709 13:15:13.515611 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet b10b16798ae24794b186d5daa625c6d1 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:15:13.896013 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:13.946457 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:13.947494 11397 raft_consensus.cc:606] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:13.954778 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:13.955973 11397 raft_consensus.cc:606] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:13.967187 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d02abb4f52449679a2415346f354340"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:13.968310 11397 raft_consensus.cc:606] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:13.971704 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:13.972294 11395 raft_consensus.cc:606] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:14.005517 11468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:14.006106 11468 raft_consensus.cc:606] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:14.009156 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:14.009671 11395 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:14.011144 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:14.012696 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:14.052913 12576 raft_consensus.cc:993] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.053392 12499 raft_consensus.cc:1081] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.054633 12118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d02abb4f52449679a2415346f354340"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:15:14.055284 12118 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.055573 12118 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.061167 12118 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.063674 12118 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:14.067939 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:14.068598 11546 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.068856 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.073410 11395 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.073845 11395 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:14.074697 11395 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.075385 11546 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.076658 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:14.082616 11395 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.086067 12584 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.096998 12499 raft_consensus.cc:993] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.098575 12556 raft_consensus.cc:1081] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.100860 12482 raft_consensus.cc:993] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.101329 12482 raft_consensus.cc:1081] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.101780 11546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:15:14.102443 11546 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.102820 11546 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.103703 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:15:14.104214 11543 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.104805 11543 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.109908 11395 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.113519 11543 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.119712 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:14.119712 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.120441 11397 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.120419 11468 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.120944 11468 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:14.121004 11543 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:14.121702 11468 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.122694 11468 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.123574 11546 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.127103 11397 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.128326 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:14.129104 12581 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.129835 12581 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:14.130755 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.131873 12584 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:14.130775 12581 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.139099 12584 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.132406 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.140517 11395 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.140976 11395 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:14.141952 11395 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.143625 11395 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.136631 11546 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:14.136811 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:14.145406 11477 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.154103 11477 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.156934 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:14.157907 12519 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.158413 12519 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:14.159022 11395 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.159288 12519 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.168305 11298 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.170074 11300 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.190091 11300 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.209023 12576 raft_consensus.cc:993] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.209465 12587 raft_consensus.cc:1081] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.210803 12118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:15:14.211215 12118 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.211442 12118 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.216799 12118 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.218725 12587 raft_consensus.cc:993] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.220016 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.220243 12587 raft_consensus.cc:1081] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:14.220288 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:14.220599 11395 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.220912 11395 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:14.220930 11477 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.219296 12118 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:14.221644 11395 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.222774 11395 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.222872 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:15:14.223421 11543 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.223742 11543 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.227324 11395 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.227342 11477 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.228451 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:14.229260 12584 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.229745 12584 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:14.230571 12584 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.231040 11543 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.232456 12576 raft_consensus.cc:993] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.232911 12576 raft_consensus.cc:1081] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:14.233765 11543 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:14.234416 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.234912 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:14.235190 12118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:15:14.235467 11477 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.235765 12118 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.236217 12118 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.235441 11395 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.236853 11395 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:14.238072 11395 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.239665 11395 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.242405 11477 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.243559 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:14.244395 12519 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.244846 12519 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:14.245682 12519 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:14.246150 12118 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.248107 12118 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:14.250828 11395 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:14.250936 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:14.265044 11543 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.254769 11300 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.251011 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:14.267765 11397 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:14.268066 11397 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:14.268781 11397 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.269982 11397 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.279817 11298 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.281030 11397 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.281860 11543 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:14.282294 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:14.283166 12584 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.283636 12584 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:14.284441 12584 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:14.292786 11300 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.553262 11397 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:14.561106 11543 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:14.561388 11366 consensus_peers.cc:597] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b61ac66d97f84c1186c6e2b1d79477b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.562916 11477 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:14.562358 12576 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:14.564082 12499 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.570804 12576 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.572255 11477 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.574187 11546 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.588845 11300 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.598865 11546 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:14.604066 11397 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.605707 12519 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.619493 11397 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.624557 12584 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:14.628149 11477 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.632519 12581 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:14.634860 11515 consensus_peers.cc:597] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12c14d0fd7ef4fbf899a4b3eebe33e5b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.645159 11395 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:14.648758 12561 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.654842 11477 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:14.656631 11397 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.660959 12581 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.666694 12604 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:14.673219 11546 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.678622 12586 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:14.677695 12561 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.687659 11469 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.707062 11543 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:14.708968 11397 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.706532 11468 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.726316 12586 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:14.736857 11515 consensus_peers.cc:597] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13474bcfb1f1443f82e90db3445bc880. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.739053 11477 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:14.743247 12519 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.745527 11395 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:14.737898 11298 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.748402 12561 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.753183 11399 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.761286 12586 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:14.761704 11469 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.785567 12519 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.787290 11397 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.802731 11395 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.804772 11397 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.812026 12584 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:14.815670 12605 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:14.815742 12581 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:14.822512 11477 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:14.825546 12561 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:14.826367 11543 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.824921 11473 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:15:14.827837 12584 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:14.856905 11348 auto_rebalancer.cc:629] Failed to schedule move for tablet 38b39fd56a8d45f7b56a9ac114b66d2b: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:15:14.867118 11297 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 2, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.013720 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:15.067430 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:15.124480 12627 ts_tablet_manager.cc:933] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:15:15.139956 12628 ts_tablet_manager.cc:933] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.147958 12627 tablet_copy_client.cc:323] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:15:15.150097 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet b61ac66d97f84c1186c6e2b1d79477b2 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:15:15.150799 12628 tablet_copy_client.cc:323] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:15:15.150804 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet b61ac66d97f84c1186c6e2b1d79477b2 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-b61ac66d97f84c1186c6e2b1d79477b2
I20260709 13:15:15.152729 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 13474bcfb1f1443f82e90db3445bc880 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:15:15.153234 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 13474bcfb1f1443f82e90db3445bc880 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-13474bcfb1f1443f82e90db3445bc880
I20260709 13:15:15.158816 11561 tablet_copy_source_session.cc:215] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:15.158836 11412 tablet_copy_source_session.cc:215] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:15.159303 11543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59af340b26264692af5490466dd69b78"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:15.160660 11543 raft_consensus.cc:606] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.163115 12628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13474bcfb1f1443f82e90db3445bc880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:15.163705 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:15.164280 11473 raft_consensus.cc:606] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:15.167179 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:15.167743 11473 raft_consensus.cc:606] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:15.171464 12627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61ac66d97f84c1186c6e2b1d79477b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:15.170042 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:15.172395 11473 raft_consensus.cc:606] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.180114 12628 tablet_copy_client.cc:806] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:15.180819 12628 tablet_copy_client.cc:670] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:15.193665 12627 tablet_copy_client.cc:806] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:15.194391 12627 tablet_copy_client.cc:670] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:15.196254 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:15.196712 11473 raft_consensus.cc:606] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:15.199258 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:15.199784 11395 raft_consensus.cc:606] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.201201 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:15.202749 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:15.204994 12628 tablet_copy_client.cc:538] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:15.210157 12627 tablet_copy_client.cc:538] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:15.215232 12628 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:15.222451 12627 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:15.227116 12633 raft_consensus.cc:993] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.227718 12633 raft_consensus.cc:1081] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.230365 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55074
I20260709 13:15:15.231069 12122 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.233032 12122 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.249754 12628 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:15.250746 12628 tablet_bootstrap.cc:492] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:15.251503 12628 ts_tablet_manager.cc:1403] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.008s
I20260709 13:15:15.254555 12628 raft_consensus.cc:359] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.255439 12628 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:15.256073 12628 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.258608 12628 ts_tablet_manager.cc:1434] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:15.256618 12627 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:15.259728 12627 tablet_bootstrap.cc:492] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:15.260447 12627 ts_tablet_manager.cc:1403] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.015s
I20260709 13:15:15.263281 12627 raft_consensus.cc:359] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.264403 12627 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:15.265070 12627 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.267990 12627 ts_tablet_manager.cc:1434] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:15:15.299784 12122 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.307577 12633 raft_consensus.cc:993] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:15.308153 12585 raft_consensus.cc:1081] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:15.309734 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:15.310007 11399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:15:15.310732 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b61ac66d97f84c1186c6e2b1d79477b2 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:15:15.310813 11399 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.311265 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b61ac66d97f84c1186c6e2b1d79477b2 on tablet b61ac66d97f84c1186c6e2b1d79477b2 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.311349 11399 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.311053 12636 ts_tablet_manager.cc:933] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.311666 11395 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.313011 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-13474bcfb1f1443f82e90db3445bc880 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:15:15.313454 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-13474bcfb1f1443f82e90db3445bc880 on tablet 13474bcfb1f1443f82e90db3445bc880 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.313468 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:15.315490 11468 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.315817 11468 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:15.316594 11468 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.317642 11468 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.318398 12636 tablet_copy_client.cc:323] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:15:15.318131 11399 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.310194 12122 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:15.319252 11395 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.320194 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:15:15.320726 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-12c14d0fd7ef4fbf899a4b3eebe33e5b
I20260709 13:15:15.325340 11468 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.325973 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:15.326521 11473 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.326829 11473 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:15.326632 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:15.327586 12584 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.327562 11473 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.328187 12584 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:15.328713 11473 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.329070 12584 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.333966 11561 tablet_copy_source_session.cc:215] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:15.338281 11399 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.339116 12636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:15.339562 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:15.340078 11543 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.343439 11473 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:15.345306 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:15.355805 12576 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.356345 12576 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:15.357374 12576 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.356382 11543 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:15.378188 11297 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.382349 12636 tablet_copy_client.cc:806] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:15.383035 12636 tablet_copy_client.cc:670] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:15.385535 11298 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.389686 12636 tablet_copy_client.cc:538] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:15.400966 12636 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:15:15.419763 12636 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:15.420851 12636 tablet_bootstrap.cc:492] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:15:15.422015 12636 ts_tablet_manager.cc:1403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.007s
I20260709 13:15:15.425055 12636 raft_consensus.cc:359] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.425890 12636 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:15:15.426537 12636 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:15.429068 12636 ts_tablet_manager.cc:1434] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:15.430727 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-12c14d0fd7ef4fbf899a4b3eebe33e5b received from {username='slave'} at 127.0.0.1:41380
I20260709 13:15:15.431114 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-12c14d0fd7ef4fbf899a4b3eebe33e5b on tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:15.458698 12561 raft_consensus.cc:993] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.459335 12604 raft_consensus.cc:1081] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.460961 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59af340b26264692af5490466dd69b78"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55076
I20260709 13:15:15.461409 12122 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.461762 12122 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.467453 12122 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.471330 12122 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.472512 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:15.473060 11399 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.473258 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:15.473812 11543 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.474126 11543 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:15.474953 11543 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.476114 11543 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.479698 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.480798 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:15.481654 12584 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.482133 12584 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:15.482349 11543 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.483029 12584 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.495790 11298 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.519186 12499 raft_consensus.cc:993] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.519687 12499 raft_consensus.cc:1081] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:15.521070 12122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:15:15.521550 12122 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.521819 12122 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.528419 12122 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.530036 12122 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.530792 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:15.530977 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:15.531492 11399 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.531580 11543 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.535020 11399 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:15.538877 12122 raft_consensus.cc:1217] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:15.537726 11399 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.543546 11399 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.546895 11543 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.548122 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:15.549145 12584 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.549609 12584 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:15.550408 12584 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:15.563469 12583 raft_consensus.cc:993] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:15.563980 12585 raft_consensus.cc:1081] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:15.565778 11546 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:15:15.566293 11546 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.566671 11546 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.572561 11546 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.574882 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:15.566249 11297 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.575477 11397 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.581192 11546 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:15.583550 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:15.584255 11473 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.584647 11473 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:15.585479 11473 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.586237 11397 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:15.588261 11473 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.588764 11516 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:15.589895 12604 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.591276 12585 raft_consensus.cc:993] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:15.591854 12583 raft_consensus.cc:1081] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:15.592129 12604 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:15.593021 12604 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.595760 11473 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:15.596653 11399 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:15.597080 11397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:15:15.603978 11397 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:15.604408 11397 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.611596 11397 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.618072 11397 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:15.620121 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:15.620275 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:15.620759 11543 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.621121 11473 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:15.621562 11473 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:15.622413 11473 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.624603 11473 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:15.636078 11297 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.638674 11543 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:15.647356 12122 raft_consensus.cc:1217] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:15.647814 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:15.648722 12576 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.649205 12576 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:15.649655 11473 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:15.650122 12576 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:15.667965 11297 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.817202 11546 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:15.818751 12499 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:15.832975 11468 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:15.834398 12499 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:15.848625 11397 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:15.851403 12619 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:15.879832 12118 raft_consensus.cc:1217] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:15.892062 11468 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:15.893463 12619 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:15.975826 11546 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:15.977133 12619 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:16.007301 11399 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:16.012095 11399 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:16.013279 12519 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:16.013207 12584 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:16.021968 12576 raft_consensus.cc:1064] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:16.022100 11546 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:16.024107 12576 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:16.028854 12620 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:16.040311 11543 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:16.041755 12499 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:16.047644 11468 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:16.051013 12499 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:16.054920 12122 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:16.057111 11395 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:16.056725 12587 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:16.076313 12620 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:16.078713 11468 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:16.090176 12581 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:16.107050 12660 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:15:16.207736 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:16.243232 12665 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:15:18.140270 10257 debug-util.cc:398] Leaking SignalData structure 0x7b080000f440 after lost signal to thread 11365
W20260709 13:15:18.141551 10257 kernel_stack_watchdog.cc:198] Thread 11365 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 592ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:15:18.206539 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08001581e0 after lost signal to thread 10255
I20260709 13:15:18.284185 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:19.142656 10257 debug-util.cc:398] Leaking SignalData structure 0x7b080000f040 after lost signal to thread 11365
W20260709 13:15:19.143263 10257 kernel_stack_watchdog.cc:198] Thread 11365 stuck at /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:249 for 592ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:15:19.632803 11468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.537s	user 0.040s	sys 0.046s
W20260709 13:15:19.633497 11468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.538s	user 0.040s	sys 0.046s
I20260709 13:15:19.646250 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 727) took 3552 ms. Trace:
I20260709 13:15:19.646685 11468 rpcz_store.cc:276] 0709 13:15:16.093378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.093820 (+   442us) service_pool.cc:224] Handling call
0709 13:15:16.094150 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.094474 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.094509 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.095017 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.634120 (+3539103us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:19.639199 (+  5079us) log.cc:844] Serialized 14 byte log entry
0709 13:15:19.645548 (+  6349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.645690 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.645772 (+    82us) raft_consensus.cc:1685] finished
0709 13:15:19.645795 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.646073 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4152,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":5340,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":4879,"thread_start_us":3548317,"threads_started":4,"wal-append.queue_time_us":237136,"wal-append.run_cpu_time_us":2082,"wal-append.run_wall_time_us":1002096}
I20260709 13:15:19.714504 12673 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:19.717267 12674 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:19.716708 12673 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:19.718452 12674 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:19.721240 12673 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:19.722195 12673 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:19.722739 12674 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:19.723143 12673 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:19.726174 12673 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:19.728004 12674 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:19.728420 12673 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:19.736696 12674 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:19.745360 12674 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:19.741798 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1439) took 3652 ms. Trace:
I20260709 13:15:19.737370 12673 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:19.750370 11546 rpcz_store.cc:276] 0709 13:15:16.089087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.089288 (+   201us) service_pool.cc:224] Handling call
0709 13:15:19.739997 (+3650709us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:19.740516 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.741044 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.741087 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:19.741166 (+    79us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:19.741252 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.741346 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.741524 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23370112}
I20260709 13:15:19.765815 12673 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:20.145254 10257 debug-util.cc:398] Leaking SignalData structure 0x7b080000fac0 after lost signal to thread 11365
W20260709 13:15:20.145825 10257 kernel_stack_watchdog.cc:198] Thread 11365 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 591ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 13:15:20.120519 12604 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:20.152858 12604 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.235606 12683 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.236692 12683 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.238410 12680 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.246064 12678 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.247176 12678 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.250737 12678 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.251199 12673 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.269587 12673 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.272352 12673 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.259418 12679 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.275648 12679 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.278298 12679 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.280673 12683 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.245190 12674 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.263418 12680 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.318320 12674 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.343417 12604 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:20.354485 12682 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:20.355459 12682 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.358265 12682 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:20.359277 12681 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.360136 12681 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.358850 12682 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.367197 12682 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.371001 12686 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.371858 12688 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.372376 12686 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.344519 12673 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.379981 12682 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:20.380685 12682 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.381475 12682 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.374217 12680 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:20.394843 12691 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.371387 12681 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.387389 12674 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:20.402272 11399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.311s	user 0.014s	sys 0.046s
I20260709 13:15:20.403807 12689 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.372884 12688 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.404680 12689 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:20.402884 11399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.312s	user 0.014s	sys 0.046s
I20260709 13:15:20.399957 12684 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.419050 12696 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.420749 12673 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:20.425823 12673 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.426368 12685 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.428473 12681 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.426779 12604 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.432499 12684 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.429490 12691 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.428346 12692 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.420269 12696 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:20.429344 12681 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.433684 12694 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.669800 12677 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.452787 12689 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.670660 12677 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.705807 12679 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.706593 12679 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.708187 12677 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:20.708892 12679 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:20.709432 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 780) took 4623 ms. Trace:
I20260709 13:15:20.453652 12683 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.713567 12687 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.713831 12697 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.710238 12674 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.710076 12673 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.715235 12689 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.715961 12679 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.453902 12604 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.708943 12692 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.427091 12685 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.009266 12682 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.226586 12700 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.987715 12688 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.197607 12620 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:21.257136 11366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.002s
W20260709 13:15:21.316015 11366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.006s	sys 0.003s
I20260709 13:15:21.235590 12710 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.315826 12620 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.316838 12710 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.066205 11299 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 226) took 1351 ms. Trace:
I20260709 13:15:21.201498 12668 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.273134 12676 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.299525 11468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f946b7973fe482681135635493ebd48"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:21.225961 12702 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.205880 12690 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.278586 12711 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.319950 11468 raft_consensus.cc:606] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:21.320449 12690 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.319340 12668 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.323895 12668 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:21.225791 11399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.047s	sys 0.100s
I20260709 13:15:21.205436 12677 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:21.324522 11399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.049s	sys 0.105s
I20260709 13:15:21.310055 12604 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.319258 12710 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.325064 12677 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.985993 12694 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.324544 12668 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.302094 12716 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.266335 12708 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.296433 12719 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.326644 12668 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:20.987973 12684 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:20.908277 12673 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:15:21.021098 11397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.994s	user 0.018s	sys 0.066s
I20260709 13:15:21.305541 12721 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.228905 12699 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:21.306102 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.002s	sys 0.019s
I20260709 13:15:20.985457 12696 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.104559 11297 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 138) took 5028 ms. Trace:
I20260709 13:15:21.312548 12722 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.908053 12674 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.272342 12678 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:21.014499 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.965s	user 0.043s	sys 0.045s
W20260709 13:15:21.024123 11331 debug-util.cc:398] Leaking SignalData structure 0x7b080003a320 after lost signal to thread 11334
I20260709 13:15:21.197889 12619 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.314841 12700 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.307595 12679 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.907639 12693 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.197258 12584 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.318132 12723 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:20.711468 12686 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.320506 12711 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.318773 12620 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.320051 12702 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.267009 12707 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.265079 12680 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.989684 12687 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.309571 12685 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.327416 12719 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.349368 12684 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.010454 12683 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:20.715984 12691 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.306481 12689 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.318681 11299 rpcz_store.cc:276] 0709 13:15:19.715224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.132396 (+417172us) service_pool.cc:224] Handling call
0709 13:15:21.065967 (+933571us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 13:15:21.168457 12675 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.333204 11297 rpcz_store.cc:276] 0709 13:15:16.076041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.157179 (+4081138us) service_pool.cc:224] Handling call
0709 13:15:21.104306 (+947127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210358,"reported_tablets":1}
I20260709 13:15:21.336673 12713 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.338238 12714 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.336697 12677 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.339314 12604 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.339919 12718 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.332510 12669 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:20.989464 12681 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:21.349861 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 5.301s	user 0.064s	sys 0.099s
I20260709 13:15:20.945784 12701 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.170295 12695 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.351833 12619 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.326730 12708 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:21.332182 11397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 5.305s	user 0.024s	sys 0.068s
I20260709 13:15:20.709936 11395 rpcz_store.cc:276] 0709 13:15:16.086077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.086181 (+   104us) service_pool.cc:224] Handling call
0709 13:15:16.086428 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:16.086605 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:20.363272 (+4276667us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.436099 (+ 72827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.440993 (+  4894us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:20.448813 (+  7820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.662253 (+213440us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.709145 (+ 46892us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4318818,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":20588,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":275,"thread_start_us":58667,"threads_started":1}
I20260709 13:15:21.182566 12704 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.335279 12690 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.341653 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:21.648617 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.648780 11398 raft_consensus.cc:606] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:21.650050 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:21.650058 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.650743 11468 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:21.651546 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.651472 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.652207 11470 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:21.652552 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.653179 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.653787 11470 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:21.654444 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.325898 12710 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.655953 12710 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.658053 12710 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.423128 12711 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.330793 12721 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:21.424649 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08002b0b00 after lost signal to thread 11426
W20260709 13:15:21.423903 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 5.37467e+06us
I20260709 13:15:21.353338 12716 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.356089 12722 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.423739 12714 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.442600 12718 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.473232 12717 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.425839 12713 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:21.485601 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 76514.9us
I20260709 13:15:21.473239 12726 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.488972 12715 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.431736 12723 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.525194 12728 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.525470 12719 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.529278 12724 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.543823 12712 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.542531 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 59) took 5457 ms. Trace:
I20260709 13:15:21.546576 12727 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.667842 12718 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.670694 12728 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.550293 12733 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.552157 12729 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.552407 12734 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.554565 12732 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.574906 12730 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.576058 12720 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:21.331700 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.002s	sys 0.019s
I20260709 13:15:21.549360 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1141) took 5456 ms. Trace:
I20260709 13:15:21.959444 11544 rpcz_store.cc:276] 0709 13:15:16.092574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.092838 (+   264us) service_pool.cc:224] Handling call
0709 13:15:21.063515 (+4970677us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:21.063926 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.334931 (+271005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.334975 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:21.335043 (+    68us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:21.344981 (+  9938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.548889 (+203908us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54001a6dd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:21.548958 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.549145 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629178112}
I20260709 13:15:21.496135 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:21.542373 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110808us
I20260709 13:15:21.994141 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:21.571987 12738 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.619030 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:21.426469 12584 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.428810 12604 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.025499 12604 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.532063 12619 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.373801 12620 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.030879 12620 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.033769 12620 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.468608 12668 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.448644 12669 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.475543 12673 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.534116 12674 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.440260 12675 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.329468 12676 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:21.623965 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:21.350580 12678 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.515686 12679 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.426569 12680 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.069639 12680 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.433320 12681 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:22.070963 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75081.4us
I20260709 13:15:21.435106 12684 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.568439 12685 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.073468 12685 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.073901 12684 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.075560 12685 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.528460 12687 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.554044 12688 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.077662 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.077729 12688 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.078361 11547 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.079917 12688 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.522477 12689 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.508252 12692 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.425351 12693 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.082700 12685 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.463757 12695 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.354570 12696 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:20.963987 12697 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.354897 12699 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.530014 12700 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.444836 12701 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.477605 12702 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:21.528379 12691 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:21.287977 12703 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.590626 12704 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.492995 12706 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.419459 12707 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.527042 12708 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.626271 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 57) took 5600 ms. Trace:
I20260709 13:15:21.627462 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:21.375324 12686 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.630626 12739 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:21.646055 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.660720 12721 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.661444 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.661712 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:21.659789 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08002ccdc0 after lost signal to thread 11501
I20260709 13:15:21.665081 12716 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.667438 12715 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.677183 12720 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.671660 11399 rpcz_store.cc:276] 0709 13:15:16.084972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.085071 (+    99us) service_pool.cc:224] Handling call
0709 13:15:16.085361 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.089741 (+  4380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.089777 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.090293 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.427358 (+4337065us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.434333 (+  6975us) log.cc:844] Serialized 14 byte log entry
0709 13:15:21.542144 (+1107811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.542281 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.542332 (+    51us) raft_consensus.cc:1685] finished
0709 13:15:21.542347 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.542476 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":689,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":45,"raft.queue_time_us":15558,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":13984,"thread_start_us":5409411,"threads_started":3,"wal-append.queue_time_us":833718,"wal-append.run_cpu_time_us":2482,"wal-append.run_wall_time_us":1177364}
I20260709 13:15:21.677564 12728 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.689656 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:21.681105 12733 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.672045 12717 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.771009 12722 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.802681 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.673163 12727 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.691169 12712 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.673079 12726 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.692785 12734 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.863859 12723 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.862171 12711 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.671175 12724 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:21.676023 12732 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.867749 12719 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.678635 12730 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:21.693282 12729 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.873961 12713 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:21.865774 11470 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:21.874818 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.878326 12714 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:21.887302 12725 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:21.890738 11368 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:41730 dropped due to backpressure: service queue is full with 50 items
I20260709 13:15:21.891068 12499 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.896827 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:21.912354 12741 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.914616 12740 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.915307 12710 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.916857 12742 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.921044 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:21.944759 12731 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.948640 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:21.961757 12744 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.962548 11550 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:21.625155 12698 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:21.972604 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:21.974259 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:21.978787 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:21.979671 12747 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:21.989696 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.000574 11548 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.011793 12584 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.017494 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.020502 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.000108 12738 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.996342 12718 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.042384 12669 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.001017 12690 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.047930 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.050189 11400 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:22.055810 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.051677 12676 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.059762 12604 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.061899 12675 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.062845 12678 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:22.061409 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67010.8us
I20260709 13:15:22.064711 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.066149 12677 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.070881 12681 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:21.504793 12683 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.079563 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.080332 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.081163 12688 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.081969 12687 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.082814 12684 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:21.506251 12694 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.195304 12499 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.196671 12692 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:22.197547 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.198197 11470 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:22.198671 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.199491 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.199246 12718 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.200248 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.200671 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.201530 12718 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.202211 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.203166 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.090209 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.203624 12733 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.088165 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.204387 12727 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.084352 12693 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.205304 12686 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.205927 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.206193 12698 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.206418 12604 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.196941 12717 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.207796 11473 raft_consensus.cc:606] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:22.207281 12114 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:22.209101 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.209386 11393 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:22.209668 11549 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.208069 12122 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:22.211112 12118 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:22.214180 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.261198 11468 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:22.214856 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.260166 11395 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:22.201593 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.221551 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:22.206146 11331 debug-util.cc:398] Leaking SignalData structure 0x7b080028ee60 after lost signal to thread 11575
I20260709 13:15:22.212327 12696 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.224422 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:22.271034 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 199769us
I20260709 13:15:22.224510 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.206393 11397 rpcz_store.cc:276] 0709 13:15:16.025790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.025899 (+   109us) service_pool.cc:224] Handling call
0709 13:15:16.026179 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.026474 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.026499 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.026923 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:15:21.538146 (+5511223us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:21.553667 (+ 15521us) log.cc:844] Serialized 14 byte log entry
0709 13:15:21.624778 (+ 71111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.626016 (+  1238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.626066 (+    50us) raft_consensus.cc:1685] finished
0709 13:15:21.626079 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.626218 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15563,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":2875,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":158,"thread_start_us":5596424,"threads_started":4,"wal-append.queue_time_us":645355,"wal-append.run_cpu_time_us":34477,"wal-append.run_wall_time_us":3106116}
I20260709 13:15:22.222656 12711 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:22.227953 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 279524us
I20260709 13:15:22.276145 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.277722 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:22.085086 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.236627 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:22.240141 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.221745 12744 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.225728 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:22.257287 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 48262us
I20260709 13:15:22.235740 12718 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.225117 12747 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.241813 12684 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.265300 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.222172 12691 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.222640 12687 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.269896 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.195303 12685 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.201622 12499 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.290601 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.217578 12690 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.243311 11542 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.259933 12716 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.282565 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.260257 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.287698 12718 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.259294 12725 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.309940 12123 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:22.263494 12742 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:22.272167 11331 debug-util.cc:398] Leaking SignalData structure 0x7b0800274d40 after lost signal to thread 11576
I20260709 13:15:22.249534 12719 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.284264 11545 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.293587 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.294135 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.294390 12729 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.294591 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.275946 12710 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.249676 12688 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.316702 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.248572 12677 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.295156 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.301255 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.214912 11368 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:41730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 64)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1390)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
I20260709 13:15:22.303959 12122 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:22.302920 11544 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.309218 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.221763 12740 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.314131 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.291268 12694 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:22.310369 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.315317 12733 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:22.245548 12706 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.263671 12731 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.304271 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.309015 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:22.318398 11546 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.221978 12741 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:22.326465 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.326766 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.310365 12716 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.373448 12716 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.374897 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.341243 12684 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.249399 12703 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.384563 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.341913 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.341190 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.311748 12738 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.357611 12604 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:22.360009 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51353.8us
I20260709 13:15:22.331382 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.331328 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.332675 12719 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.440970 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.441936 11477 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:22.453603 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.455145 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.330962 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.332841 12710 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.331817 12688 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.364765 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.459981 11468 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:22.365396 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.365619 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.372340 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.316992 12718 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.437618 12685 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.375519 11551 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.365842 12734 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.353813 12733 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.363267 12727 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.466343 12727 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.466414 12747 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.466545 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.466800 12727 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.467200 11468 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:22.468108 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.468726 11468 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:22.469147 12730 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.469517 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.342151 12707 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.353726 12694 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.387524 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:22.341277 12697 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.347940 12726 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.374327 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.420110 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.458537 12691 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.490309 11473 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:22.490937 12715 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:22.419054 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08003222e0 after lost signal to thread 12147
I20260709 13:15:22.491274 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.492748 11551 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:22.420728 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.422755 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.410470 11476 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:22.458901 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:22.460784 11548 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
W20260709 13:15:22.461961 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 89801.6us
I20260709 13:15:22.498903 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.354898 12681 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.453779 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.465153 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.464388 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:22.498698 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78043.8us
I20260709 13:15:22.483100 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.459712 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.456681 12701 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.463757 12696 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.464615 12708 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.471794 12733 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:22.506886 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.483880 12694 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:22.470901 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:22.443590 12706 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:22.508725 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50865.4us
I20260709 13:15:22.509471 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.514401 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:22.514859 12742 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.489789 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.487260 12698 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.529904 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:22.531193 12693 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.531620 12693 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.531963 12693 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.508603 12715 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.487455 12744 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.508088 12690 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.532495 11368 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:41728 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:22.509008 12697 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:22.517903 12708 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.513101 12681 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.508826 12727 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:22.518810 11469 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:22.519148 12711 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.543519 12715 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.544449 12696 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:22.527772 12741 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.549965 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:22.545688 12708 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.553085 12677 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:22.662299 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 114158us
I20260709 13:15:22.543512 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:22.627509 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:22.667080 11395 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:22.661672 11368 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:41730 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1391)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 791)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
W20260709 13:15:22.639410 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8d00 after lost signal to thread 10255
I20260709 13:15:22.545837 12697 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:22.627485 12695 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:22.539573 12725 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:22.627033 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.619958 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:22.705482 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.545935 11550 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:22.855520 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.709947 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:22.872423 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 341433us
I20260709 13:15:22.873535 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:22.671288 12730 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:22.627142 12715 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.566666 12681 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:22.927994 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:22.629781 12720 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:22.928062 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08001d71e0 after lost signal to thread 11334
I20260709 13:15:22.929013 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:22.671885 12741 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:23.056362 11516 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.597s (SENT) [suppressed 14 similar messages]
I20260709 13:15:23.062261 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.063503 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.064531 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.071973 11548 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:22.911356 12703 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:22.928192 12698 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:23.079171 11498 debug-util.cc:398] Leaking SignalData structure 0x7b080025c660 after lost signal to thread 11426
W20260709 13:15:23.083953 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 155003us
I20260709 13:15:23.134269 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:23.084125 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 156215us
I20260709 13:15:23.053571 12681 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.129598 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.135794 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:23.143493 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.148478 11399 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
W20260709 13:15:23.149121 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 797) took 1509 ms (client timeout 1840 ms). Trace:
I20260709 13:15:23.076719 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:22.914175 12731 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:23.153303 11399 rpcz_store.cc:269] 0709 13:15:21.639613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.930338 (+1290725us) service_pool.cc:224] Handling call
0709 13:15:23.149053 (+218715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.911512 12697 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:23.162146 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.139979 12695 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.140138 12696 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.130213 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:23.016551 12730 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:23.164742 12733 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.621s	user 0.003s	sys 0.008s
I20260709 13:15:23.151270 12703 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:23.179447 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.176040 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.153024 12740 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.183437 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.183525 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:23.185047 12712 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:23.187909 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.186908 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1185) took 1309 ms. Trace:
I20260709 13:15:23.174114 12725 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:23.170899 12695 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:23.171311 12696 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:23.174149 12727 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.197338 11473 rpcz_store.cc:276] 0709 13:15:21.878487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.493764 (+615277us) service_pool.cc:224] Handling call
0709 13:15:22.501768 (+  8004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.508033 (+  6265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.508063 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.508109 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:22.508144 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.508206 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.186744 (+678538us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b540042a4d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.186850 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17726848}
W20260709 13:15:23.181445 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800154880 after lost signal to thread 11501
I20260709 13:15:23.180078 12730 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.209800 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.173396 12741 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:23.207201 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.228529 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.205770 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:23.205139 12718 consensus_meta.cc:287] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.716s	user 0.006s	sys 0.005s
I20260709 13:15:23.230870 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.229357 11543 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:23.181118 12116 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:23.234021 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1200) took 1314 ms. Trace:
I20260709 13:15:23.235350 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1457) took 1140 ms. Trace:
I20260709 13:15:23.207827 12695 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.201166 12725 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:23.242722 11545 rpcz_store.cc:276] 0709 13:15:22.095355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.230674 (+1135319us) service_pool.cc:224] Handling call
0709 13:15:23.234750 (+  4076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.234951 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.235016 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.235077 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:23.235132 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.235187 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.235285 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:23.240278 12693 consensus_meta.cc:287] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.000s
W20260709 13:15:23.243321 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 791) took 1610 ms (client timeout 1985 ms). Trace:
W20260709 13:15:23.259166 11399 rpcz_store.cc:269] 0709 13:15:21.633063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.167979 (+1534916us) service_pool.cc:224] Handling call
0709 13:15:23.243249 (+ 75270us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:23.260100 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800365080 after lost signal to thread 11575
W20260709 13:15:23.260468 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08003650a0 after lost signal to thread 11576
I20260709 13:15:23.262471 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.189481 12733 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:23.208074 12696 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.259711 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1201) took 1339 ms. Trace:
I20260709 13:15:23.265762 11397 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:23.254714 12725 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:23.201458 12703 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:23.261570 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800365960 after lost signal to thread 12147
I20260709 13:15:23.270483 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:23.246330 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 64) took 1603 ms (client timeout 1823 ms). Trace:
W20260709 13:15:23.267494 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 802) took 1377 ms (client timeout 1577 ms). Trace:
W20260709 13:15:23.272193 11397 rpcz_store.cc:269] 0709 13:15:21.890090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.230393 (+1340303us) service_pool.cc:224] Handling call
0709 13:15:23.267426 (+ 37033us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:23.269114 11368 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50180 dropped due to backpressure: service queue is full with 50 items [suppressed 2 similar messages]
I20260709 13:15:23.266366 11473 rpcz_store.cc:276] 0709 13:15:21.920304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.221724 (+1301420us) service_pool.cc:224] Handling call
0709 13:15:23.252155 (+ 30431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.259269 (+  7114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.259313 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.259370 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.259413 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.259493 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.259643 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.277117 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:23.271536 11402 rpcz_store.cc:269] 0709 13:15:21.642491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.193097 (+1550606us) service_pool.cc:224] Handling call
0709 13:15:23.246250 (+ 53153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.273074 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50180 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
W20260709 13:15:23.269723 11472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.037s	sys 0.061s
I20260709 13:15:23.248100 12718 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:23.288206 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1388) took 1397 ms (client timeout 1782 ms). Trace:
W20260709 13:15:23.288611 11399 rpcz_store.cc:269] 0709 13:15:21.886973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.262903 (+1375930us) service_pool.cc:224] Handling call
0709 13:15:23.284789 (+ 21886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.291587 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.293854 12718 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:23.287490 11472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.037s	sys 0.061s
I20260709 13:15:23.261967 12693 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:23.287768 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50180 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.314254 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50180 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 72)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.337417 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50180 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
W20260709 13:15:23.356680 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 794) took 1722 ms (client timeout 2100 ms). Trace:
W20260709 13:15:23.357599 11397 rpcz_store.cc:269] 0709 13:15:21.633920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.280142 (+1646222us) service_pool.cc:224] Handling call
0709 13:15:23.356598 (+ 76456us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.363354 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 37) took 1413 ms. Trace:
I20260709 13:15:23.363773 11473 rpcz_store.cc:276] 0709 13:15:21.942724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.282322 (+1339598us) service_pool.cc:224] Handling call
0709 13:15:23.296110 (+ 13788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.299523 (+  3413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.299556 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.355393 (+ 55837us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:23.355459 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.355551 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.355734 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.357159 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41728 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1395)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.371675 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:23.372860 11393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1368) took 1501 ms (client timeout 1645 ms). Trace:
W20260709 13:15:23.373243 11393 rpcz_store.cc:269] 0709 13:15:21.871843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.371308 (+1499465us) service_pool.cc:224] Handling call
0709 13:15:23.372782 (+  1474us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:23.375420 11542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1147) took 1383 ms (client timeout 1671 ms). Trace:
W20260709 13:15:23.375833 11542 rpcz_store.cc:269] 0709 13:15:21.992154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.000323 (+  8169us) service_pool.cc:224] Handling call
0709 13:15:23.375222 (+1374899us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54003e99d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.375341 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64021248}
I20260709 13:15:23.377491 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.378093 11395 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
W20260709 13:15:23.378531 11395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1391) took 1490 ms (client timeout 1652 ms). Trace:
W20260709 13:15:23.378834 11395 rpcz_store.cc:269] 0709 13:15:21.887971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.377252 (+1489281us) service_pool.cc:224] Handling call
0709 13:15:23.378483 (+  1231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.380119 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:23.381064 11398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1387) took 1494 ms (client timeout 1501 ms). Trace:
W20260709 13:15:23.381477 11398 rpcz_store.cc:269] 0709 13:15:21.886196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.379803 (+1493607us) service_pool.cc:224] Handling call
0709 13:15:23.380988 (+  1185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.372200 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41728 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1395)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.384068 12724 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.246963 11476 rpcz_store.cc:276] 0709 13:15:21.919958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.187908 (+1267950us) service_pool.cc:224] Handling call
0709 13:15:23.188384 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.199606 (+ 11222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.205474 (+  5868us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.209220 (+  3746us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.223631 (+ 14411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.230627 (+  6996us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.233960 (+  3333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.397450 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1150) took 1342 ms. Trace:
I20260709 13:15:23.404119 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 180) took 1308 ms. Trace:
I20260709 13:15:23.406183 11546 rpcz_store.cc:276] 0709 13:15:22.055378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.055539 (+   161us) service_pool.cc:224] Handling call
0709 13:15:23.397287 (+1341748us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400404a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.397395 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253244672}
I20260709 13:15:23.410358 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 304) took 1137 ms. Trace:
I20260709 13:15:23.403167 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:23.410588 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.374195 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.408068 12122 rpcz_store.cc:276] 0709 13:15:22.088015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.260768 (+172753us) service_pool.cc:224] Handling call
0709 13:15:23.396141 (+1135373us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b5400745254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.396257 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11783936}
I20260709 13:15:23.410722 12118 rpcz_store.cc:276] 0709 13:15:22.267225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.292103 (+ 24878us) service_pool.cc:224] Handling call
0709 13:15:22.318373 (+ 26270us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:22.394082 (+ 75709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.394273 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.394299 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.410187 (+ 15888us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.412365 (+  2178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.412428 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.404282 (+991854us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b540047c050. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.404393 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165013632}
I20260709 13:15:23.411361 11399 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
W20260709 13:15:23.430372 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1366) took 1734 ms (client timeout 2061 ms). Trace:
W20260709 13:15:23.430891 11399 rpcz_store.cc:269] 0709 13:15:21.696232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.397099 (+1700867us) service_pool.cc:224] Handling call
0709 13:15:23.430317 (+ 33218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.422796 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 782) took 1846 ms. Trace:
I20260709 13:15:23.427399 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.427560 12731 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.433156 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.359954 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 305) took 1045 ms. Trace:
I20260709 13:15:23.383785 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41728 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.434322 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:23.397822 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.434521 11396 rpcz_store.cc:276] 0709 13:15:21.575839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.592697 (+ 16858us) service_pool.cc:224] Handling call
0709 13:15:21.594429 (+  1732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.315818 (+721389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.315865 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.315941 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:22.315972 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.422305 (+1106333us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54003101d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:23.422373 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.422563 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649555328}
I20260709 13:15:23.428140 11469 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:23.445716 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.448153 11399 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
W20260709 13:15:23.448746 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1378) took 1565 ms (client timeout 1989 ms). Trace:
W20260709 13:15:23.449142 11399 rpcz_store.cc:269] 0709 13:15:21.883791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.431748 (+1547957us) service_pool.cc:224] Handling call
0709 13:15:23.448680 (+ 16932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.428432 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.452776 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:23.433315 12731 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:23.458966 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.435885 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 788) took 1805 ms. Trace:
I20260709 13:15:23.443166 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:23.466127 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 331964us
I20260709 13:15:23.469046 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:23.469427 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 333729us
W20260709 13:15:23.469671 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 413425us
I20260709 13:15:23.498360 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.500653 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.501444 11544 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:23.505724 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.506606 11545 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:23.507763 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.508185 11367 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:23.509135 12724 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.509508 12724 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:23.509884 12724 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.515264 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.515913 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.516899 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.517504 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.518392 11546 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:23.519301 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.519495 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.518429 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:23.533684 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.534842 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.463655 12731 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.464193 12114 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:23.547516 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.563359 11477 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:23.520860 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.463070 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.566494 11546 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:23.460691 12739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.072s	sys 0.202s
I20260709 13:15:23.644578 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1468) took 1132 ms. Trace:
I20260709 13:15:23.462615 11393 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:23.645237 11543 rpcz_store.cc:276] 0709 13:15:22.511988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.241738 (+729750us) service_pool.cc:224] Handling call
0709 13:15:23.552185 (+310447us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:23.552419 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.644109 (+ 91690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.644150 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.644208 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.644264 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.644349 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.644507 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15229696}
I20260709 13:15:23.509778 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:23.757074 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 247350us
I20260709 13:15:23.757742 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:23.758037 12696 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:23.655107 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.759342 11476 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
W20260709 13:15:23.655354 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 366589us
I20260709 13:15:23.562368 12733 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:23.755743 12715 consensus_meta.cc:287] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
I20260709 13:15:23.763195 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.669695 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.761819 12758 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:23.669935 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 304435us
W20260709 13:15:23.771705 11398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1374) took 1890 ms (client timeout 1912 ms). Trace:
I20260709 13:15:23.772294 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.772591 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:23.773828 11515 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:23.682397 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 431116us
I20260709 13:15:23.464578 11394 rpcz_store.cc:276] 0709 13:15:21.628263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.629335 (+  1072us) service_pool.cc:224] Handling call
0709 13:15:21.629624 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.256407 (+626783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.257743 (+  1336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.262997 (+  5254us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.269693 (+  6696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.408978 (+1139285us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54003a70d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:23.411734 (+  2756us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.434043 (+ 22309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393309568}
I20260709 13:15:23.875787 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.652479 12703 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:23.683990 12122 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:23.775836 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:23.965364 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 189576us
I20260709 13:15:23.966071 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:23.584991 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1392) took 1665 ms (client timeout 1504 ms). Trace:
W20260709 13:15:23.969640 11400 rpcz_store.cc:269] 0709 13:15:21.888255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.330255 (+1442000us) service_pool.cc:224] Handling call
0709 13:15:23.553505 (+223250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.654825 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.761359 12756 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.032577 12756 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.033079 12756 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.032583 11401 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:23.776124 12766 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:24.035022 11401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1370) took 2155 ms (client timeout 1637 ms). Trace:
I20260709 13:15:24.035487 12766 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.776346 12757 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:23.771718 12758 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.037014 12766 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:24.037114 12758 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.037447 12766 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.762739 12760 raft_consensus.cc:1064] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
W20260709 13:15:23.772073 11398 rpcz_store.cc:269] 0709 13:15:21.881396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.755726 (+1874330us) service_pool.cc:224] Handling call
0709 13:15:23.771651 (+ 15925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.783670 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:23.792409 11368 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:41728 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 70)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 73)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1401)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48200 (request call id 7)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807)
I20260709 13:15:23.824105 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:23.827859 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 287803us
I20260709 13:15:24.039639 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 312) took 1675 ms. Trace:
W20260709 13:15:23.829664 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1361) took 2166 ms (client timeout 2241 ms). Trace:
I20260709 13:15:23.837200 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.040001 12122 rpcz_store.cc:276] 0709 13:15:22.363938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.539694 (+1175756us) service_pool.cc:224] Handling call
0709 13:15:23.622487 (+ 82793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.656102 (+ 33615us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:24.021946 (+365844us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.039329 (+ 17383us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.039378 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:24.039418 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.039476 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.039585 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":15963,"threads_started":1,"wal-append.queue_time_us":11627}
W20260709 13:15:24.041527 11396 rpcz_store.cc:269] 0709 13:15:21.663290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.458455 (+1795165us) service_pool.cc:224] Handling call
0709 13:15:23.829586 (+371131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.041949 11395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1367) took 2262 ms (client timeout 2073 ms). Trace:
W20260709 13:15:23.662678 11393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1377) took 1772 ms (client timeout 1888 ms). Trace:
I20260709 13:15:24.042557 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.042810 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 187) took 1678 ms. Trace:
W20260709 13:15:24.042737 11393 rpcz_store.cc:269] 0709 13:15:21.883496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.373926 (+1490430us) service_pool.cc:224] Handling call
0709 13:15:23.656258 (+282332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.043268 11396 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
W20260709 13:15:23.858178 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 798) took 2218 ms (client timeout 2164 ms). Trace:
W20260709 13:15:23.864472 11366 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.044s	sys 0.077s
I20260709 13:15:23.782133 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:23.704576 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 751) took 1097 ms. Trace:
I20260709 13:15:23.760293 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.093930 11476 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:23.778700 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 752) took 1438 ms. Trace:
I20260709 13:15:24.096055 11477 rpcz_store.cc:276] 0709 13:15:22.339860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.776694 (+1436834us) service_pool.cc:224] Handling call
0709 13:15:23.776964 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.777117 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.777142 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.777179 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:23.777209 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.777262 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.778654 (+  1392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.098402 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1211) took 1594 ms. Trace:
I20260709 13:15:23.751732 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1192) took 1867 ms. Trace:
I20260709 13:15:23.747207 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.725833 11473 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:23.762039 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:23.779749 11515 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:24.112270 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 332563us
I20260709 13:15:24.113477 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:24.114383 12765 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:23.884078 11394 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:24.115069 12765 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:24.115242 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1381) took 2230 ms (client timeout 2166 ms). Trace:
I20260709 13:15:24.115437 12765 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:24.115684 11394 rpcz_store.cc:269] 0709 13:15:21.884615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.875458 (+1990843us) service_pool.cc:224] Handling call
0709 13:15:24.115177 (+239719us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:23.717269 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 220579us
I20260709 13:15:24.119177 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.119910 12703 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.120275 12703 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:23.821234 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:23.806411 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1465) took 1379 ms. Trace:
I20260709 13:15:23.725833 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1160) took 1658 ms. Trace:
I20260709 13:15:23.812117 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1444) took 1720 ms. Trace:
W20260709 13:15:23.839102 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 434137us
W20260709 13:15:23.755659 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 292799us
I20260709 13:15:24.129068 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.129472 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:23.791744 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:23.703363 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 272) took 1728 ms. Trace:
I20260709 13:15:23.745570 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 278) took 1766 ms. Trace:
I20260709 13:15:23.683281 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 307) took 1353 ms. Trace:
I20260709 13:15:23.435757 12115 rpcz_store.cc:276] 0709 13:15:22.287316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.292181 (+  4865us) service_pool.cc:224] Handling call
0709 13:15:23.263258 (+971077us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:23.270749 (+  7491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.290527 (+ 19778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.295032 (+  4505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.295110 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:23.295176 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.295249 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.302390 (+  7141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8215936}
I20260709 13:15:24.140239 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 313) took 1747 ms. Trace:
I20260709 13:15:24.140617 12115 rpcz_store.cc:276] 0709 13:15:22.393343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.139341 (+1745998us) service_pool.cc:224] Handling call
0709 13:15:24.139682 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.139884 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.139919 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.139963 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:24.139999 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.140060 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.140183 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:23.750685 12116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 186) took 1386 ms. Trace:
I20260709 13:15:23.751670 12686 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:23.797884 12687 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:23.786648 12681 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.004s
I20260709 13:15:24.148492 12681 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:23.768100 12696 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.711303 12693 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:23.739696 12694 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.130s	user 0.008s	sys 0.006s
I20260709 13:15:23.872498 12699 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:23.650034 12704 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:23.755169 12708 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.046s	user 0.008s	sys 0.002s
I20260709 13:15:24.154534 12708 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.156666 12708 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:23.764678 12716 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.391s	user 0.001s	sys 0.012s
I20260709 13:15:24.157872 12716 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.159050 12112 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.159941 12716 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.160596 11475 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:23.767256 12715 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.164592 12119 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:24.165333 12715 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.763530 12725 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:24.166131 12119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 181) took 1855 ms (client timeout 1985 ms). Trace:
W20260709 13:15:24.166587 12119 rpcz_store.cc:269] 0709 13:15:22.310601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.331070 (+ 20469us) service_pool.cc:224] Handling call
0709 13:15:23.183371 (+852301us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:24.165901 (+982530us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54004de254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:24.166064 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238792192}
W20260709 13:15:24.167625 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.167725 12725 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:24.168416 12123 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:23.761468 12730 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.000s
I20260709 13:15:24.170598 12730 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.172665 12730 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:23.754714 12695 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:24.174948 12695 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:23.768286 12697 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.003s
I20260709 13:15:24.176083 12697 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.177070 11468 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.177996 12697 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:24.178148 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1198) took 2258 ms. Trace:
I20260709 13:15:23.748965 12724 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.178471 11468 rpcz_store.cc:276] 0709 13:15:21.919258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.548263 (+629005us) service_pool.cc:224] Handling call
0709 13:15:22.626231 (+ 77968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.177401 (+1551170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.177931 (+   530us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400214154. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.178094 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3566217088}
I20260709 13:15:24.180369 12724 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.789877 12731 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.181964 12118 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:24.182775 12731 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:23.768523 12732 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:23.761989 12733 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.185029 12733 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.185412 12733 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.775349 12736 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:23.629069 12739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.072s	sys 0.203s
W20260709 13:15:23.683735 12742 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.035s
I20260709 13:15:23.935527 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.193226 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1362) took 2529 ms (client timeout 2076 ms). Trace:
W20260709 13:15:24.193583 11402 rpcz_store.cc:269] 0709 13:15:21.663613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.402333 (+1738720us) service_pool.cc:224] Handling call
0709 13:15:23.935386 (+533053us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:24.193160 (+257774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36356864}
I20260709 13:15:24.194808 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.195595 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 817) took 1534 ms (client timeout 1689 ms). Trace:
I20260709 13:15:23.971473 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:23.966827 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:24.196722 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 229944us
I20260709 13:15:24.197463 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:24.195912 11402 rpcz_store.cc:269] 0709 13:15:22.661179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.194556 (+1533377us) service_pool.cc:224] Handling call
0709 13:15:24.195497 (+   941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.198661 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.199131 12732 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.199527 12732 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.199846 12732 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:23.981276 12718 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.201382 12718 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:24.201771 12718 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.988559 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1199) took 2069 ms. Trace:
I20260709 13:15:24.203343 11469 rpcz_store.cc:276] 0709 13:15:21.919608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.677346 (+757738us) service_pool.cc:224] Handling call
0709 13:15:23.058302 (+380956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.451006 (+392704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.988276 (+537270us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:23.988490 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790044544}
W20260709 13:15:24.035424 11401 rpcz_store.cc:269] 0709 13:15:21.879789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.409140 (+1529351us) service_pool.cc:224] Handling call
0709 13:15:24.034919 (+625779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.204218 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1196) took 2286 ms. Trace:
I20260709 13:15:24.204581 11475 rpcz_store.cc:276] 0709 13:15:21.917831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.518658 (+600827us) service_pool.cc:224] Handling call
0709 13:15:22.543722 (+ 25064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.160966 (+1617244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.204005 (+ 43039us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540028bc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.204164 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718080384}
I20260709 13:15:24.205029 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.205868 11401 raft_consensus.cc:2393] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:24.205911 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 306) took 1896 ms. Trace:
W20260709 13:15:24.039958 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 256337us
I20260709 13:15:24.206346 12112 rpcz_store.cc:276] 0709 13:15:22.309065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.644178 (+335113us) service_pool.cc:224] Handling call
0709 13:15:22.813813 (+169635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.159415 (+1345602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.205684 (+ 46269us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540021dc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.205857 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2983433472}
W20260709 13:15:24.041100 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1393) took 2152 ms (client timeout 2121 ms). Trace:
W20260709 13:15:24.207475 11399 rpcz_store.cc:269] 0709 13:15:21.888536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.823827 (+1935291us) service_pool.cc:224] Handling call
0709 13:15:24.041031 (+217204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.044065 11397 rpcz_store.cc:269] 0709 13:15:21.639929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.857947 (+2218018us) service_pool.cc:211] Skipping call since client already timed out
0709 13:15:23.858086 (+   139us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:15:24.208336 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.208446 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.043938 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.044173 11366 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.045s	sys 0.077s
I20260709 13:15:24.043296 12122 rpcz_store.cc:276] 0709 13:15:22.364814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.042038 (+1677224us) service_pool.cc:224] Handling call
0709 13:15:24.042322 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.042503 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.042527 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.042564 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:24.042597 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.042658 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.042764 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.046000 12757 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:24.042326 11395 rpcz_store.cc:269] 0709 13:15:21.779439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.836921 (+2057482us) service_pool.cc:224] Handling call
0709 13:15:24.041887 (+204966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.045591 12760 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:24.043848 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1386) took 2157 ms (client timeout 2239 ms). Trace:
I20260709 13:15:24.059293 12756 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.055066 12758 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.064893 12766 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:24.102334 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 320240us
I20260709 13:15:24.098843 11477 rpcz_store.cc:276] 0709 13:15:22.504307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.097346 (+1593039us) service_pool.cc:224] Handling call
0709 13:15:24.097760 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.097998 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.098038 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.098100 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:24.098141 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.098210 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.098351 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.099834 11472 rpcz_store.cc:276] 0709 13:15:21.884425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.510364 (+625939us) service_pool.cc:224] Handling call
0709 13:15:22.510730 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.511121 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.511157 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.515201 (+  4044us) log.cc:844] Serialized 26 byte log entry
0709 13:15:23.365948 (+850747us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.366009 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.366078 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:23.751444 (+385366us) raft_consensus.cc:1685] finished
0709 13:15:23.751472 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.751629 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1871,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":569088,"thread_start_us":850065,"threads_started":1,"wal-append.queue_time_us":1038732}
I20260709 13:15:24.104830 12678 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.105782 11470 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:24.210417 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 974450us
I20260709 13:15:24.102907 11471 rpcz_store.cc:276] 0709 13:15:22.339562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.398724 (+1059162us) service_pool.cc:224] Handling call
0709 13:15:23.414115 (+ 15391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.431681 (+ 17566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.431721 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.436782 (+  5061us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:23.436838 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.436917 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.437055 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.116748 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.211645 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 808) took 1701 ms (client timeout 1765 ms). Trace:
W20260709 13:15:24.212067 11394 rpcz_store.cc:269] 0709 13:15:22.510474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.116477 (+1606003us) service_pool.cc:224] Handling call
0709 13:15:24.211569 (+ 95092us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.206369 11401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 811) took 1695 ms (client timeout 1892 ms). Trace:
I20260709 13:15:24.212711 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.213157 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.214037 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 806) took 1704 ms (client timeout 2064 ms). Trace:
W20260709 13:15:24.214411 11394 rpcz_store.cc:269] 0709 13:15:22.509896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.212877 (+1702981us) service_pool.cc:224] Handling call
0709 13:15:24.213979 (+  1102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.122363 11547 rpcz_store.cc:276] 0709 13:15:22.437174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.233609 (+796435us) service_pool.cc:224] Handling call
0709 13:15:23.805596 (+571987us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4e9c 00007f8da67f4a5b 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:23.805800 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.806021 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.806053 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.806101 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.806134 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.806200 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.806343 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4371200}
I20260709 13:15:24.122849 11550 rpcz_store.cc:276] 0709 13:15:22.067772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.483723 (+415951us) service_pool.cc:224] Handling call
0709 13:15:22.511549 (+ 27826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.519812 (+  8263us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:23.197135 (+677323us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.725206 (+528071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.725283 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:23.725346 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.725611 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.725739 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":210034,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":499252,"threads_started":2,"wal-append.queue_time_us":508739}
I20260709 13:15:24.215695 12699 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.115841 12765 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.216100 12699 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:24.216522 12699 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.216985 12758 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.125980 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.218438 11545 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:15:24.126730 11551 rpcz_store.cc:276] 0709 13:15:22.091318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.494192 (+402874us) service_pool.cc:224] Handling call
0709 13:15:23.397407 (+903215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.404279 (+  6872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.404305 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.404369 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:23.404399 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.404473 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.811895 (+407422us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400404a50. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:23.812050 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076604672}
I20260709 13:15:24.131189 12764 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.130604 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.131922 12117 rpcz_store.cc:276] 0709 13:15:21.975324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.975462 (+   138us) service_pool.cc:224] Handling call
0709 13:15:21.975761 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.262963 (+287202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.265265 (+  2302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.269722 (+  4457us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.280448 (+ 10726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.653692 (+1373244us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540043d854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:23.670236 (+ 16544us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.700755 (+ 30519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605994368}
I20260709 13:15:24.133545 12113 rpcz_store.cc:276] 0709 13:15:21.979046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.979326 (+   280us) service_pool.cc:224] Handling call
0709 13:15:21.979685 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.411977 (+432292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.412012 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.412069 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.412106 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.745286 (+1333180us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54000c3fd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:23.745345 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.745497 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957969280}
W20260709 13:15:24.134912 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 62360us
W20260709 13:15:24.222411 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 91913.9us
I20260709 13:15:24.134125 12114 rpcz_store.cc:276] 0709 13:15:22.316076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.360213 (+1044137us) service_pool.cc:224] Handling call
0709 13:15:23.443350 (+ 83137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.643738 (+200388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.669372 (+ 25634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.224597 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:24.225381 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:24.225988 12770 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.226115 11515 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:24.226425 12770 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.227232 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:24.142616 12116 rpcz_store.cc:276] 0709 13:15:22.364546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.749692 (+1385146us) service_pool.cc:224] Handling call
0709 13:15:23.750028 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.750300 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.750329 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.750407 (+    78us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:23.750443 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.750500 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.750630 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.229594 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.144346 12771 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.150602 12681 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.152784 12694 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.169836 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 308) took 1853 ms. Trace:
W20260709 13:15:24.196763 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 813) took 1665 ms (client timeout 1523 ms). Trace:
W20260709 13:15:24.242837 11400 rpcz_store.cc:269] 0709 13:15:22.531611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.970398 (+1438787us) service_pool.cc:224] Handling call
0709 13:15:24.196696 (+226298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.243237 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.244033 11471 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:24.207005 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.045857 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.244946 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.245365 11398 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:24.245939 11398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 814) took 1714 ms (client timeout 1591 ms). Trace:
I20260709 13:15:24.246330 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:24.246397 11398 rpcz_store.cc:269] 0709 13:15:22.531889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.039401 (+1507512us) service_pool.cc:224] Handling call
0709 13:15:24.245855 (+206454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.247234 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.247751 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.209021 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 810) took 1698 ms (client timeout 1960 ms). Trace:
W20260709 13:15:24.248751 11399 rpcz_store.cc:269] 0709 13:15:22.510979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.208076 (+1697097us) service_pool.cc:224] Handling call
0709 13:15:24.208967 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.213006 11401 rpcz_store.cc:269] 0709 13:15:22.511258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.204739 (+1693481us) service_pool.cc:224] Handling call
0709 13:15:24.206313 (+  1574us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.209021 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 812) took 1677 ms (client timeout 1982 ms). Trace:
I20260709 13:15:24.120635 12703 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.124495 11516 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.215258 12762 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.227255 12757 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:24.230007 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 100640us
I20260709 13:15:24.235284 12764 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.241540 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.241629 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.242352 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.243952 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.249789 11397 rpcz_store.cc:269] 0709 13:15:22.531268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.208238 (+1676970us) service_pool.cc:224] Handling call
0709 13:15:24.208974 (+   736us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.228178 12742 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.015s	sys 0.036s
I20260709 13:15:24.226817 12770 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.248574 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.219874 12704 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:24.252962 12762 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:24.280495 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.280782 12762 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.248585 11398 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:24.239566 12123 rpcz_store.cc:276] 0709 13:15:22.316430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.462325 (+1145895us) service_pool.cc:224] Handling call
0709 13:15:24.168203 (+705878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.168819 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.169648 (+   829us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54004de250. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:24.169779 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1578038016}
W20260709 13:15:24.254171 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47277.7us
I20260709 13:15:24.255349 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.283696 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.284508 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.236418 12768 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.232355 11402 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:24.236186 12696 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.264410 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.288213 11477 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:24.287392 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 815) took 1755 ms (client timeout 2133 ms). Trace:
W20260709 13:15:24.289618 11402 rpcz_store.cc:269] 0709 13:15:22.532168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.229361 (+1697193us) service_pool.cc:224] Handling call
0709 13:15:24.287317 (+ 57956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.290792 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.291527 11402 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:24.292662 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.269277 12732 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.270268 11542 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:24.271447 12769 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.234972 11393 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:24.264406 11395 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
W20260709 13:15:24.302261 11395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 809) took 1791 ms (client timeout 2121 ms). Trace:
W20260709 13:15:24.217617 11396 rpcz_store.cc:269] 0709 13:15:21.885941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.042296 (+2156355us) service_pool.cc:224] Handling call
0709 13:15:24.043790 (+  1494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.305292 12736 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.247234 12725 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.251936 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:24.278707 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 52655.7us
I20260709 13:15:24.307025 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.279011 12775 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.279747 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.279902 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.259991 12757 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.288265 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1203) took 2087 ms. Trace:
I20260709 13:15:24.320554 11473 rpcz_store.cc:276] 0709 13:15:22.200434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.364666 (+1164232us) service_pool.cc:224] Handling call
0709 13:15:23.365193 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:23.365551 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.365575 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:23.366765 (+  1190us) log.cc:844] Serialized 540 byte log entry
0709 13:15:23.725699 (+358934us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:24.136835 (+411136us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.288001 (+151166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.288070 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:24.288109 (+    39us) raft_consensus.cc:1685] finished
0709 13:15:24.288119 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.288215 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8621,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":481221,"threads_started":2,"wal-append.queue_time_us":384336}
I20260709 13:15:24.289384 12771 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.246579 12739 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:15:24.248984 12695 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:15:24.326042 12695 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.326378 12695 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:24.305729 12736 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.326833 12736 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.260325 12766 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.261682 12681 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.287055 12716 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.286861 11400 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:24.246433 12724 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:15:24.300510 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.301285 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.349133 11399 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:24.349622 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 67) took 1066 ms. Trace:
I20260709 13:15:24.349892 11399 rpcz_store.cc:276] 0709 13:15:23.283234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.278102 (+994868us) service_pool.cc:224] Handling call
0709 13:15:24.349568 (+ 71466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.350729 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:24.302785 11395 rpcz_store.cc:269] 0709 13:15:22.510720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.217913 (+1707193us) service_pool.cc:224] Handling call
0709 13:15:24.302174 (+ 84261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.303217 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 68) took 1012 ms. Trace:
I20260709 13:15:24.351750 11402 rpcz_store.cc:276] 0709 13:15:23.283571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.292383 (+1008812us) service_pool.cc:224] Handling call
0709 13:15:24.295540 (+  3157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.303970 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.304697 12704 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.309059 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.314366 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.316243 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.313476 12777 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.305716 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.319584 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.288522 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.294716 12768 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.294466 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:24.323896 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 199478us
I20260709 13:15:24.294888 12696 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.324105 12694 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.302505 12769 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.330098 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.331333 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.307292 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.335115 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.335597 12116 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.327275 12695 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:24.310681 11393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 803) took 1890 ms (client timeout 1759 ms). Trace:
I20260709 13:15:24.351491 12122 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:24.330900 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.451606 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:24.310191 12725 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.324026 12775 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:24.411099 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 122636us
W20260709 13:15:24.353130 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 819) took 1185 ms (client timeout 1529 ms). Trace:
W20260709 13:15:24.368026 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59034.3us
I20260709 13:15:24.345461 12681 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:24.446250 11393 rpcz_store.cc:269] 0709 13:15:22.419158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.043633 (+1624475us) service_pool.cc:224] Handling call
0709 13:15:24.309332 (+265699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.459898 11393 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.460058 12681 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.460561 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 792) took 2827 ms. Trace:
I20260709 13:15:24.460925 11398 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
W20260709 13:15:24.428865 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 142670us
I20260709 13:15:24.413913 12768 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.367106 12778 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:24.410670 12779 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.435364 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.432348 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.465116 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1399) took 1020 ms. Trace:
I20260709 13:15:24.357988 12699 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.430135 12756 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.429730 12696 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.448760 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.450421 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.451213 12115 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:24.432696 12769 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.446373 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.454510 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.453272 11397 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:24.444255 12119 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.465523 11399 rpcz_store.cc:276] 0709 13:15:23.444904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.423015 (+978111us) service_pool.cc:224] Handling call
0709 13:15:24.465058 (+ 42043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.457484 12775 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.435801 12718 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.525882 12765 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.464216 12778 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:24.527910 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 193209us
W20260709 13:15:24.524528 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57160.6us
I20260709 13:15:24.500659 12695 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.531651 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.464788 12779 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:24.456617 11400 rpcz_store.cc:269] 0709 13:15:23.166007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.243699 (+1077692us) service_pool.cc:224] Handling call
0709 13:15:24.351415 (+107716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.502933 12732 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.533110 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.544296 11468 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:24.534210 12756 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.534622 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:24.535059 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 80633.3us
I20260709 13:15:24.551326 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1397) took 1168 ms. Trace:
I20260709 13:15:24.551705 11397 rpcz_store.cc:276] 0709 13:15:23.383081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.280803 (+897722us) service_pool.cc:224] Handling call
0709 13:15:24.319453 (+ 38650us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:24.551166 (+231713us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b54003a7ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:24.551267 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207562368}
I20260709 13:15:24.538193 12694 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.562139 12694 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.562559 12694 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.460923 11393 rpcz_store.cc:276] 0709 13:15:21.633352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.459272 (+2825920us) service_pool.cc:224] Handling call
0709 13:15:24.459705 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.460308 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.460506 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.538785 12725 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.567198 11470 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:24.569527 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 795) took 2934 ms. Trace:
I20260709 13:15:24.569839 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 62) took 2928 ms. Trace:
I20260709 13:15:24.570820 11397 rpcz_store.cc:276] 0709 13:15:21.641129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.568841 (+2927712us) service_pool.cc:224] Handling call
0709 13:15:24.569143 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.569288 (+   145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.569314 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.569378 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:24.569408 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.569457 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.569786 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.573058 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 63) took 2931 ms. Trace:
I20260709 13:15:24.539731 12765 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:24.542281 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.550822 12119 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:24.537876 11396 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:24.554685 12778 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.557665 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1168) took 1058 ms. Trace:
I20260709 13:15:24.560969 12777 raft_consensus.cc:1064] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:24.572847 12736 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.570217 11399 rpcz_store.cc:276] 0709 13:15:21.635195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.545931 (+2910736us) service_pool.cc:224] Handling call
0709 13:15:24.568773 (+ 22842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.569034 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.569080 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.569144 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:24.569193 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.569272 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.569417 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.470309 12693 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:15:24.574110 11397 rpcz_store.cc:276] 0709 13:15:21.642095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.572315 (+2930220us) service_pool.cc:224] Handling call
0709 13:15:24.572607 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.572762 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.572787 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.572825 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:24.572853 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.572903 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.573009 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.539371 12704 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:24.543483 12779 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.608843 11400 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.613652 11399 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.543171 12724 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:24.620174 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 100749us
I20260709 13:15:24.581390 11542 rpcz_store.cc:276] 0709 13:15:23.498928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.499920 (+   992us) service_pool.cc:224] Handling call
0709 13:15:23.500251 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.298347 (+798096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.557422 (+259075us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54001a6dd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.557607 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1746668416}
I20260709 13:15:24.585520 12697 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:24.621343 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 175049us
I20260709 13:15:24.621699 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1364) took 2934 ms. Trace:
I20260709 13:15:24.624825 11399 rpcz_store.cc:276] 0709 13:15:21.681885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.592230 (+2910345us) service_pool.cc:224] Handling call
0709 13:15:24.613447 (+ 21217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.614173 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.614384 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.624140 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.626027 11399 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.626684 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.627810 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.628664 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.610672 12704 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:24.631071 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79169.7us
I20260709 13:15:24.631709 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.632835 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.612720 12778 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.635223 11367 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.637960 12768 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.629400 12731 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.641454 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1365) took 2941 ms. Trace:
I20260709 13:15:24.641674 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.641829 11399 rpcz_store.cc:276] 0709 13:15:21.693666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.625596 (+2931930us) service_pool.cc:224] Handling call
0709 13:15:24.625878 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.629814 (+  3936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.633758 (+  3944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.619977 12724 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.646157 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.646626 11399 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.646687 12788 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.647485 12790 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.648526 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.650770 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.650945 11516 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.630532 12774 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.654549 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1369) took 2770 ms. Trace:
I20260709 13:15:24.650492 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.654558 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:24.655028 11399 rpcz_store.cc:276] 0709 13:15:21.879399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.642604 (+2763205us) service_pool.cc:224] Handling call
0709 13:15:24.642885 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.649392 (+  6507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.649561 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.633704 12716 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.655457 11477 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:24.648139 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.660120 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.620913 12757 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.667647 12777 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.669358 11402 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.642757 12731 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.652601 12724 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.670672 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.651818 12687 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.654099 12764 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.642463 12786 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.661338 12781 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.659301 12716 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.663700 12766 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.682226 12694 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:24.684986 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.644439 12703 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.686414 12716 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.670884 12762 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.690955 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.677842 12731 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.689085 12718 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:24.687719 12733 consensus_meta.cc:287] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.502s	user 0.004s	sys 0.006s
I20260709 13:15:24.685544 12781 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.684661 12732 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.688107 12765 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.686837 12766 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.676620 12696 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.682776 12786 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:24.715372 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.694545 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1372) took 2813 ms. Trace:
I20260709 13:15:24.695878 12764 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.703594 12781 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.702026 12699 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.723249 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.704622 12732 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:24.705196 12765 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:24.711316 12766 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.728410 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.698529 12733 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.728415 12694 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.731424 12732 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.716153 11401 rpcz_store.cc:276] 0709 13:15:21.880708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.673771 (+2793063us) service_pool.cc:224] Handling call
0709 13:15:24.674098 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.689098 (+ 15000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.692097 (+  2999us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.694252 (+  2155us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:24.694298 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.694367 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.694488 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.743299 12123 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.736013 12768 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.733767 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.739362 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:24.716545 12786 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.733858 12765 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.740633 12736 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.749009 12693 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:24.763018 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 92389.4us
W20260709 13:15:24.765864 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 115163us
I20260709 13:15:24.750506 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.775039 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.761653 12733 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.783202 11474 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.783816 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.767333 12696 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.784721 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.788556 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 793) took 3154 ms. Trace:
I20260709 13:15:24.796118 11402 rpcz_store.cc:276] 0709 13:15:21.633623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.468865 (+2835242us) service_pool.cc:224] Handling call
0709 13:15:24.469147 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.673542 (+204395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.788285 (+114743us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001e00d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.788488 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431551872}
I20260709 13:15:24.797214 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.797782 11402 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:24.798841 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.770970 12693 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:24.801783 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.789520 12698 raft_consensus.cc:993] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:24.804121 11402 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:24.806253 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.806538 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.807931 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 60) took 3167 ms. Trace:
I20260709 13:15:24.811870 11471 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.816411 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.816790 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.817013 12122 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:24.817198 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.818842 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.820449 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:24.821247 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 179631us
I20260709 13:15:24.821945 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.822021 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.822954 12698 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.823378 12698 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:24.823734 12698 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.823764 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.823941 12786 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.825587 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.826071 12786 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.828649 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.796152 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.833101 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:24.830596 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.832110 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:24.827054 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.844086 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.812608 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.849175 12758 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.853135 12704 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.814502 12757 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.861595 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.827670 12696 raft_consensus.cc:1081] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:24.854674 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:24.837672 12681 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:24.838081 12703 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:24.833129 12742 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.858014 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.887560 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.888136 11545 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.812825 11400 rpcz_store.cc:276] 0709 13:15:21.640473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.553474 (+2913001us) service_pool.cc:224] Handling call
0709 13:15:24.584993 (+ 31519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.612274 (+ 27281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.807714 (+195440us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b54003d40d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:24.807853 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68478208}
I20260709 13:15:24.881305 12778 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.846850 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.889513 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.891151 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:24.882956 12770 consensus_meta.cc:287] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 13:15:24.857344 11476 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:24.894503 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.895541 12778 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:24.902894 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.903820 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.894565 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.861595 12762 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:24.865489 12681 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:24.892398 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.906090 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.880774 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:24.904842 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.924311 11546 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:24.905848 12778 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.928416 12775 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.929644 11551 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:24.895401 11547 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:24.900323 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.931667 11549 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:24.931871 12708 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:24.932263 12708 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:24.909881 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.933199 12708 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.934580 12708 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.934826 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.935576 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1371) took 3055 ms. Trace:
I20260709 13:15:24.936429 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.937295 12785 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.937690 12785 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:24.938024 12785 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.900493 11393 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.905499 12770 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:24.904055 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08001f4760 after lost signal to thread 10252
I20260709 13:15:24.934639 12758 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:24.940805 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.939823 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.916009 12704 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:24.911664 12681 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.935913 11399 rpcz_store.cc:276] 0709 13:15:21.880088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.655828 (+2775740us) service_pool.cc:224] Handling call
0709 13:15:24.656088 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.671700 (+ 15612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.671733 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.671775 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:24.671804 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.935378 (+263574us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b540030b454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.935417 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.935525 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15050496}
I20260709 13:15:24.941826 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.890391 12757 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:15:24.955796 12774 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.957015 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.957983 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.958626 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.958747 11544 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.960469 11395 raft_consensus.cc:606] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.960907 11395 raft_consensus.cc:614] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:24.961440 12758 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:24.961822 12758 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:24.961864 12698 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.962728 12758 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.953277 11551 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.951619 12716 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.954352 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:24.951790 12696 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:24.955929 12757 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:24.951768 11474 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:24.965742 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:24.967190 11398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1400) took 1516 ms (client timeout 1516 ms). Trace:
W20260709 13:15:24.976408 11398 rpcz_store.cc:269] 0709 13:15:23.451298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.282183 (+830885us) service_pool.cc:224] Handling call
0709 13:15:24.967041 (+684858us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400231c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:24.967137 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257865472}
I20260709 13:15:24.977579 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:24.978235 11398 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:24.949622 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:24.895767 12693 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:24.979773 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.980857 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.980906 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:24.981894 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.982197 12715 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.983706 12770 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.984236 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 61) took 3343 ms. Trace:
I20260709 13:15:24.984609 11393 rpcz_store.cc:276] 0709 13:15:21.640731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.564988 (+2924257us) service_pool.cc:224] Handling call
0709 13:15:24.565324 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.946406 (+381082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.983978 (+ 37572us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400230854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:24.984174 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745707776}
I20260709 13:15:24.985754 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:24.987574 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:24.988019 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:24.940253 12779 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.988786 12765 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.989156 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.989742 11469 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.990234 12779 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:24.991891 11477 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:24.993036 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.993902 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:24.946740 11545 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:24.968387 11544 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:24.996793 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:24.997493 11544 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.973095 11471 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
W20260709 13:15:24.950518 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9540 after lost signal to thread 10255
I20260709 13:15:25.000639 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.002388 11471 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.002521 11473 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.002758 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:24.964570 12795 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.005239 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.006455 11545 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.970890 12758 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.006824 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.008683 11545 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:25.020673 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 49207.7us
I20260709 13:15:25.024587 11367 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.001621 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:24.996384 12699 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.010972 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:24.975849 12757 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.043502 11398 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:24.945195 12781 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.023931 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.028916 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.010546 12695 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.051808 12733 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:25.052073 12724 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.052594 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.052855 12733 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.053216 12733 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.053783 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.054177 12724 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.054104 12733 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.055019 11397 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.055269 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.042557 12725 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.056361 12781 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.056195 12757 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:24.997722 12765 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.049414 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.069087 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.069896 12716 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.070210 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.007884 12681 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.071256 11542 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.072847 12732 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.034883 12766 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.076673 12778 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.077557 12766 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.078568 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.091078 12732 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.051196 12772 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.047910 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.092931 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.093997 11545 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:25.061100 12757 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:25.095788 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.096004 11547 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.055299 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:25.094869 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.062906 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:25.047950 12769 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 13:15:25.102468 12769 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.103747 11394 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.104671 12769 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.064980 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:25.105001 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 818) took 2103 ms (client timeout 1861 ms). Trace:
W20260709 13:15:25.105409 11394 rpcz_store.cc:269] 0709 13:15:23.002041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.256506 (+1254465us) service_pool.cc:224] Handling call
0709 13:15:25.104788 (+848282us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400348354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:25.104935 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550178688}
I20260709 13:15:25.106055 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.106482 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.106547 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:25.107307 11401 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.108453 11366 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.061214 12792 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:25.124022 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.125056 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.125701 11548 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.029374 11474 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.030133 12112 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:25.079514 11402 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.050902 12116 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:25.079769 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08001ccae0 after lost signal to thread 11334
I20260709 13:15:25.079872 12678 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.080021 12795 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.080545 12697 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.037956 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:15:25.156165 11543 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:25.156574 11543 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.081692 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1375) took 3200 ms. Trace:
I20260709 13:15:25.157262 11397 rpcz_store.cc:276] 0709 13:15:21.881782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.768528 (+2886746us) service_pool.cc:224] Handling call
0709 13:15:24.782972 (+ 14444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.055388 (+272416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.081468 (+ 26080us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540014a0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:25.081638 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625112832}
I20260709 13:15:25.083837 12686 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.084288 12779 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:25.059190 12699 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.161047 12693 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.160444 11300 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:25.086872 12733 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.163924 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.164582 11397 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.165441 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.087564 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:25.010771 12788 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:15:25.088382 11475 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.166958 12788 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.167356 12788 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.167482 11469 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:25.090595 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.033375 12687 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.008023 12739 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.169098 11469 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.171589 11470 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.166628 11397 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.093698 12731 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.078789 12778 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:25.178501 12796 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:15:25.065651 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.064579 12698 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.150234 12678 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.114180 11545 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.115861 12085 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.125589 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:25.126541 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:25.031072 12113 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:25.023525 12777 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:25.130436 12792 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.132531 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:25.135179 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.136389 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:25.187573 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51290.1us
I20260709 13:15:25.137678 12732 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:25.189821 12732 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.190084 12732 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:25.127779 12769 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.143262 11544 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.102896 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.190959 12687 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.191147 12119 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.144124 11551 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.191394 12687 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.192255 12687 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.193181 12687 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.195037 11516 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:25.140419 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1373) took 3259 ms. Trace:
W20260709 13:15:25.196676 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.143770 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.194438 12792 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.197654 12777 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.198292 12786 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.198722 12786 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:25.198560 12792 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.199167 12786 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.194108 11468 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:25.148268 11548 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:25.148599 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 90583.7us
I20260709 13:15:25.149324 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.161417 12693 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:25.086520 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48200 (request call id 7) took 1578 ms. Trace:
I20260709 13:15:25.206022 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.197666 12794 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.203774 11395 rpcz_store.cc:276] 0709 13:15:23.507832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.451448 (+943616us) service_pool.cc:224] Handling call
0709 13:15:25.086309 (+634861us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9913853 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:25.086426 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168996224}
W20260709 13:15:25.209913 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.177121 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1376) took 3295 ms. Trace:
I20260709 13:15:25.175225 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.210553 11402 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.107188 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.210366 11397 rpcz_store.cc:276] 0709 13:15:21.883168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.166218 (+3283050us) service_pool.cc:224] Handling call
0709 13:15:25.166473 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.176896 (+ 10423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.177069 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.180488 11550 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:25.182560 11398 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.110869 11367 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:25.185097 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 69343us
W20260709 13:15:25.213426 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 69746.5us
I20260709 13:15:25.159155 12697 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.190302 12732 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:25.176275 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 121063us
I20260709 13:15:25.219934 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.215965 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.215623 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.224052 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.217073 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.186291 12678 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.226440 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:25.240888 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.225675 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.228415 12123 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:25.276499 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50883.1us
W20260709 13:15:25.208819 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4a80 after lost signal to thread 11426
I20260709 13:15:25.318079 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:25.319665 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 93328.3us
I20260709 13:15:25.317310 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1380) took 3392 ms. Trace:
I20260709 13:15:25.320564 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:25.235824 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 124993us
I20260709 13:15:25.320817 11395 rpcz_store.cc:276] 0709 13:15:21.884340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.208487 (+3324147us) service_pool.cc:224] Handling call
0709 13:15:25.275991 (+ 67504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.276146 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.276170 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.276212 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:25.276239 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.276291 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.276391 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:25.279187 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62173us
I20260709 13:15:25.212617 12765 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.116s
I20260709 13:15:25.328020 11399 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.328172 12731 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.328685 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1382) took 3443 ms. Trace:
I20260709 13:15:25.329033 12775 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.331683 12699 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.333117 12777 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.333218 11394 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.209647 12788 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.211570 12686 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:15:25.335080 12788 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.329035 11399 rpcz_store.cc:276] 0709 13:15:21.884902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.275359 (+3390457us) service_pool.cc:224] Handling call
0709 13:15:25.327846 (+ 52487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.328456 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.328634 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.337242 11397 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.217892 12774 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.276551 12698 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.339936 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1383) took 3454 ms. Trace:
I20260709 13:15:25.316265 12795 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.218108 12697 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.354873 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.274865 12801 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.341895 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.218346 12769 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.325099 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:25.211637 12781 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:15:25.343377 11397 rpcz_store.cc:276] 0709 13:15:21.885169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.336802 (+3451633us) service_pool.cc:224] Handling call
0709 13:15:25.337100 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.339744 (+  2644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.339889 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.274670 12792 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.278295 12715 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.316606 12762 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.274796 12778 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.358521 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.358575 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.359001 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.338896 12768 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.339216 12794 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:25.405551 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 181576us
I20260709 13:15:25.371209 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:25.376407 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.408633 11397 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.375873 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.405284 12686 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.406354 12739 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.410437 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.348836 12785 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.348727 12708 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:25.363260 12772 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.383603 12693 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.363032 12757 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:25.385167 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45317.1us
I20260709 13:15:25.362931 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:25.388221 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.388895 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.397679 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.401635 11402 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.356608 12795 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.424659 11399 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.408339 12802 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:25.403446 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44959.2us
I20260709 13:15:25.428711 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.402352 12786 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:25.404902 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46468.2us
I20260709 13:15:25.491539 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.492558 12786 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.492712 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.494231 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.494479 11393 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:25.359690 12801 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.409344 11395 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.495623 12801 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.496302 12772 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.496603 11395 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.496781 12772 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.497212 12772 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.379766 11396 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:25.498032 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:25.405280 12799 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:15:25.413045 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.200171 11401 rpcz_store.cc:276] 0709 13:15:21.881056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.744721 (+2863665us) service_pool.cc:224] Handling call
0709 13:15:25.107123 (+362402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.107761 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.140209 (+ 32448us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400348350. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:25.140359 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832668928}
W20260709 13:15:25.403684 11423 debug-util.cc:398] Leaking SignalData structure 0x7b0800341ae0 after lost signal to thread 11501
I20260709 13:15:25.413677 12739 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.403163 12797 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.407788 12773 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.404028 12768 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.406174 12777 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.423770 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1384) took 3538 ms. Trace:
I20260709 13:15:25.380036 12774 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:15:25.404075 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46560.8us
I20260709 13:15:25.412217 12715 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.400686 11543 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.501852 12117 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.427721 12116 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.431864 12785 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.520648 11543 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:25.432333 11394 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.432737 11400 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.506300 12773 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:25.433467 12756 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:25.429009 12757 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.427977 12795 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.434321 12769 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:25.434345 12693 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.436002 12724 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.436503 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.436404 12698 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:25.450060 12732 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.469069 12762 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.532954 12773 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.407143 12794 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.534333 12773 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.495023 12686 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.495155 12708 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.498368 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.498633 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 805) took 2989 ms. Trace:
I20260709 13:15:25.497293 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 800) took 3607 ms. Trace:
I20260709 13:15:25.500099 11399 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:25.503998 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:25.505618 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.508035 11398 rpcz_store.cc:276] 0709 13:15:21.885444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.423040 (+3537596us) service_pool.cc:224] Handling call
0709 13:15:25.423304 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.423468 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.423494 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.423541 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:25.423566 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.423616 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.423720 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.510710 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.538348 11398 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.536155 11396 rpcz_store.cc:276] 0709 13:15:22.509575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.497926 (+2988351us) service_pool.cc:224] Handling call
0709 13:15:25.498200 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.498348 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.498374 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.498413 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:25.498442 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.498493 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.498588 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.424269 12712 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:15:25.537030 12708 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.507648 12768 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.425653 12778 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:15:25.550432 12786 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.550729 12698 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.550915 12786 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.551178 12698 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.522620 12739 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.518271 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:25.520866 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 804) took 3055 ms. Trace:
I20260709 13:15:25.428339 12802 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:25.527067 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8080 after lost signal to thread 11575
W20260709 13:15:25.535305 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 98877.7us
I20260709 13:15:25.536269 11395 rpcz_store.cc:276] 0709 13:15:21.889528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.496080 (+3606552us) service_pool.cc:224] Handling call
0709 13:15:25.496406 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.497057 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.497237 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.551807 12769 consensus_queue.cc:1243] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:25.537284 11402 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.508364 12797 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.522514 12715 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.542256 11547 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.544091 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:25.544341 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.575433 12696 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:25.575963 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.576483 12696 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.576577 12697 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.535895 12686 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:25.542671 12708 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.547528 12117 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:25.547710 11396 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.544013 12724 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.555896 12758 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.556104 11400 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.556766 12778 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.566241 12802 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.555996 11399 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.546310 12762 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.551982 12779 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.568713 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1385) took 3683 ms. Trace:
I20260709 13:15:25.585450 12686 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.577188 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.552750 12731 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.546474 11398 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:25.577040 12696 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.581211 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.577554 12769 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.560118 11393 rpcz_store.cc:276] 0709 13:15:22.465605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.497457 (+3031852us) service_pool.cc:224] Handling call
0709 13:15:25.497766 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.500685 (+  2919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.501839 (+  1154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.502924 (+  1085us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:25.505872 (+  2948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.509847 (+  3975us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.519646 (+  9799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.580964 12777 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.583309 12732 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.584165 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:25.609875 11423 debug-util.cc:398] Leaking SignalData structure 0x7b0800261500 after lost signal to thread 12147
I20260709 13:15:25.534579 12794 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:25.625816 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44686.8us
I20260709 13:15:25.597281 12697 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.557250 12678 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.599372 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.618080 11402 rpcz_store.cc:276] 0709 13:15:21.885690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.425052 (+3539362us) service_pool.cc:224] Handling call
0709 13:15:25.537104 (+112052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.565968 (+ 28864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.568497 (+  2529us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000424d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:25.568654 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257049344}
I20260709 13:15:25.557446 12757 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.620819 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.620908 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:25.581820 12733 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.584734 12712 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:25.592947 12806 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.604120 12758 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.627467 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.631577 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.643586 11396 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:25.643988 12716 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.644152 12785 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.644546 12756 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.599547 12786 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.644966 12756 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.645550 12786 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.645857 12756 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.630013 12777 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.598446 12772 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.650966 12777 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.643616 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:25.601444 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 103148us
I20260709 13:15:25.595842 12797 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.625660 12805 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.638733 12085 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.615445 12116 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
W20260709 13:15:25.639858 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.659019 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.638025 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.671546 11395 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.671851 11395 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.672075 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.623324 12731 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:25.594587 12781 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:15:25.609009 12778 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.599112 12768 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.656353 11367 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.652813 12772 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.629091 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:25.624495 12769 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:25.683552 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.629649 12724 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:15:25.644547 12716 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.657658 12712 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.678937 11395 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.628293 12739 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.712595 12119 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.670070 12756 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.672946 12804 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:25.691263 12808 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 13:15:25.734042 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43621.1us
I20260709 13:15:25.673421 12731 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:25.714821 12807 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:15:25.673067 12764 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:25.737670 11395 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.739933 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.657559 12806 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.713124 12781 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.722573 12792 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.714461 12678 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.716038 12797 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.713389 12813 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:15:25.759893 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:25.675462 12805 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.658898 12697 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 13:15:25.744288 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 72561.7us
I20260709 13:15:25.748992 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.746825 12777 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:15:25.781055 12777 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.781452 12777 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.782328 12777 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.783701 12777 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:25.786012 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.660972 12795 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.724824 12733 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:25.713254 12696 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.750166 11401 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.777824 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:25.780107 11393 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.715197 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:25.737381 12788 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:15:25.750284 12757 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.750447 12781 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.702263 12812 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:15:25.792248 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.724658 12810 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:15:25.808422 12693 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.808856 12693 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:25.809206 12693 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.810108 12781 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.811275 12781 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.761695 12762 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:15:25.763262 12778 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.815240 12778 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.815680 12797 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.801204 12698 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.817615 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 820) took 1023 ms. Trace:
I20260709 13:15:25.817833 12698 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.818048 11401 rpcz_store.cc:276] 0709 13:15:24.793904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.521405 (+727501us) service_pool.cc:224] Handling call
0709 13:15:25.532704 (+ 11299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.807320 (+274616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.817372 (+ 10052us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003a7ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:25.817557 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474490112}
I20260709 13:15:25.798841 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:25.806780 12686 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:25.823191 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.827107 11395 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:25.820086 11472 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.830440 11543 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.831110 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:25.831301 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.831401 11397 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.761085 12774 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:15:25.735844 12758 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.763768 12775 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.743495 12716 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.805560 12678 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.799253 12708 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:25.833312 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.792286 12704 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.833779 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.757658 12764 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:25.839414 12123 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.842334 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:25.833781 12768 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.833959 12797 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.761999 12801 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:25.841984 12766 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.847916 12696 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.866572 12686 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:25.843480 12678 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:25.865970 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 807) took 3355 ms. Trace:
I20260709 13:15:25.836359 12762 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:15:25.897145 11397 rpcz_store.cc:276] 0709 13:15:22.510190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.501212 (+2991022us) service_pool.cc:224] Handling call
0709 13:15:25.501486 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.834202 (+332716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.865715 (+ 31513us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400231c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:25.865904 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753297792}
I20260709 13:15:25.866808 12699 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:25.898427 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 220357us
I20260709 13:15:25.902616 11548 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.907097 11548 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.869009 12704 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.888989 12785 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.800510 12769 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.855672 12715 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:25.856650 12810 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:15:25.887449 12774 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:15:25.898275 12758 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.919449 12786 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:15:25.915917 12686 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:15:25.898065 12812 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:15:25.927110 12804 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.902040 12699 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.966355 12699 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.967195 12757 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.967743 12757 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:25.967873 12699 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.969369 11545 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.899756 12766 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:25.971357 12693 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.971647 12766 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.973476 12693 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.919030 12785 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:25.930814 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62691.3us
I20260709 13:15:25.911526 11547 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:25.936447 12814 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:15:25.936677 12795 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.939416 12764 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.939775 12797 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.940212 12810 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:25.849032 12806 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.944238 12801 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.947777 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.933425 12712 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:15:25.957103 12697 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.957286 12731 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:25.928639 12802 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.959743 12704 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.963693 12708 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:25.964248 12678 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:25.965188 12762 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.978427 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:25.929749 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:25.952782 12804 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:26.103744 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.107120 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 124759us
W20260709 13:15:26.116947 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 138661us
I20260709 13:15:25.933549 12715 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:26.097509 12811 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.268s	user 0.000s	sys 0.002s
I20260709 13:15:25.903440 12805 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:26.112865 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 270090us
I20260709 13:15:26.111408 12693 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.996598 12724 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
I20260709 13:15:26.118448 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:26.107872 12813 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.104409 12697 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.116400 12762 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:26.130908 12813 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.131210 12816 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:26.130173 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.119481 12712 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:26.124243 12779 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.005s
I20260709 13:15:26.124480 12693 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.169927 12678 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.172314 12816 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:26.174778 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 259532us
W20260709 13:15:26.196266 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66760.8us
I20260709 13:15:25.989674 12819 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.191s
I20260709 13:15:26.131474 12794 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:26.173357 12772 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.172989 12708 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:15:26.207053 12830 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.247159 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.246815 12816 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.248664 11474 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.169696 12821 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.199309 12779 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.192081 12724 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:26.277129 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.278119 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.278843 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.279630 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.279940 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.280486 11543 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:26.280575 11551 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:26.253639 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 90776.3us
I20260709 13:15:26.279471 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.282438 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.283403 11544 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.285229 11544 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.286878 11544 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.210353 12772 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.288836 11546 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.289487 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.257068 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.290027 11472 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:26.297726 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.291404 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.291584 12693 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:26.292356 12824 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:15:26.292907 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.295797 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.291584 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.176488 12697 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.301657 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.306231 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.303689 12816 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.287834 12724 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.305613 12693 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.305212 12772 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:26.309725 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 188151us
I20260709 13:15:26.310205 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.310230 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:26.304322 12777 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.000s
I20260709 13:15:26.299444 12732 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:26.313148 11473 raft_consensus.cc:2376] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2. Re-sending same reply.
I20260709 13:15:26.298965 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:26.316893 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.317914 11551 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:26.322502 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.342010 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.309007 12779 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:26.309821 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.312151 11547 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.374256 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.375691 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.378973 11546 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:26.349659 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.287997 12813 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.384088 12731 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:26.392732 11546 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.387720 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.349467 12732 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.393523 11542 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.394148 11472 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:26.386251 12698 consensus_meta.cc:287] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.006s
I20260709 13:15:26.398437 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.399250 11472 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.401351 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:26.385731 12772 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.389374 12715 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:26.381497 12779 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.423974 12724 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:26.424724 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.425859 12732 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:26.426951 12765 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:26.438438 11550 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:26.420311 11469 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:26.441749 11546 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.455137 12779 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:26.457340 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.457892 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.483721 11545 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.482215 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.486091 11543 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:26.458315 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.459111 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.459126 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.459431 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.462000 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:26.463984 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:26.465162 12792 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.000s
W20260709 13:15:26.475760 12781 consensus_meta.cc:287] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.008s
W20260709 13:15:26.516247 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.520138 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 52693.8us
I20260709 13:15:26.522845 11397 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.475111 12765 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:26.529492 11548 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:26.482177 12813 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.519254 12792 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:26.530350 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.573603 12765 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.574784 12765 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.528594 12758 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:26.578023 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.494501 12724 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:15:26.496039 12704 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:15:26.587687 12774 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.613664 12774 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.615550 12819 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:26.615888 12774 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:26.601646 12715 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:26.588196 12758 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.592072 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:26.618353 11475 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.610725 11468 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:26.610927 11471 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.605165 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:26.616835 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:26.615950 12819 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:26.622156 12768 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.628008 12686 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:26.628764 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.623796 11476 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.623929 12758 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:26.631714 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.632026 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.636571 11475 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:26.626677 12819 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.639516 12819 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.628419 12686 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.637786 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.642879 11543 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:26.642892 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:25.933758 12681 consensus_queue.cc:1175] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e9c2603c0bd4d25b713649b3890c46b" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:26.647424 12835 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.647876 12835 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.648643 12835 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.628288 12715 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:15:26.649773 12835 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:26.650940 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.651054 11473 raft_consensus.cc:606] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:26.651644 11473 raft_consensus.cc:614] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:26.653178 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.655395 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.656868 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.630957 12804 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.675709 12699 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.528079 12779 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:26.642113 12794 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.658751 12781 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.711827 12704 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.712388 12704 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.683822 12832 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.681663 12724 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.722268 11470 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:26.721261 12724 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.723448 12724 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.702054 12779 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.727717 12715 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:26.728101 12758 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.728302 12715 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.728631 12758 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.731201 12715 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:26.730873 12758 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:26.705142 12794 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:26.738950 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.732723 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.742072 11468 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:26.741700 11476 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:26.745018 12768 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.745081 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:26.753973 12794 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:26.755087 12768 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:26.755534 12768 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:26.756494 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.762413 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.763227 11468 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:26.763908 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.765153 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:26.755900 12768 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.717305 12686 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.755957 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:26.772727 12822 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.773120 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:26.773622 12686 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.774658 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:26.770437 12779 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:26.776261 12794 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.776116 12785 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.776674 12794 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.777022 12794 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.777060 12704 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:26.777426 12785 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.778316 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:26.773159 12822 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:26.784021 12822 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.784538 12724 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:26.795097 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.782639 12768 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:26.786362 12795 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.785373 12822 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.832407 12795 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.861924 11548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:26.862586 11548 raft_consensus.cc:606] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.864516 11395 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.865523 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.866089 11395 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:26.866143 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.867630 11398 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.867738 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.866922 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.868515 11397 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:26.868438 11395 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:26.859278 12795 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.872850 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:26.873371 11546 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.873438 11543 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:26.873683 11551 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:26.875849 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 219073us
I20260709 13:15:26.876658 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.877043 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.878301 11393 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.878646 11542 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.878235 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:26.879635 12797 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:15:26.880481 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.881120 11393 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:26.880828 12758 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:26.882412 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.883333 11393 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:26.883571 12832 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.883648 12781 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.884526 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:26.886780 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:26.887929 11393 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:26.889344 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1217) took 1064 ms. Trace:
I20260709 13:15:26.889747 11550 rpcz_store.cc:276] 0709 13:15:25.825340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.825456 (+   116us) service_pool.cc:224] Handling call
0709 13:15:26.889184 (+1063728us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400404054. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:26.889285 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153546496}
I20260709 13:15:26.915614 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:26.869240 12839 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:26.922070 12839 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.087731 12724 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:27.085016 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.035918 12832 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:27.094586 12832 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:27.095503 12765 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.096165 12765 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.164175 12819 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:27.164808 12819 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.164827 12781 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:27.167205 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:27.168051 12123 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:27.183270 12832 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:27.192399 12834 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.193145 12834 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.098773 12765 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:27.257364 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.353s	user 0.041s	sys 0.029s
W20260709 13:15:27.257874 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.353s	user 0.042s	sys 0.030s
W20260709 13:15:27.258195 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 1.35364e+06us
W20260709 13:15:27.292243 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.266665 12834 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:27.590003 12806 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.610524 12805 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:27.352962 12794 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.612972 12819 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:27.706444 12806 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.708853 12805 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:27.959635 12769 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.961956 12805 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:27.963945 12806 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:27.964640 12806 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.965237 12806 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:27.934906 12785 consensus_meta.cc:287] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.157s	user 0.014s	sys 0.000s
W20260709 13:15:27.992574 12850 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
W20260709 13:15:27.997299 12795 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.121s	user 0.005s	sys 0.005s
I20260709 13:15:27.997745 12795 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.000046 12795 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:28.001535 12822 consensus_meta.cc:287] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.157s	user 0.012s	sys 0.000s
I20260709 13:15:28.005913 12855 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.006454 12855 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.003633 12794 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.063365 12806 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.065454 12855 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.066567 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:28.067214 11473 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.076845 11473 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:28.077411 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1266) took 1202 ms. Trace:
I20260709 13:15:28.077872 11473 rpcz_store.cc:276] 0709 13:15:26.874544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.874730 (+   186us) service_pool.cc:224] Handling call
0709 13:15:28.077305 (+1202575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.087529 12757 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.089959 12802 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.095073 12762 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.096024 12762 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.118194 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 67) took 1435 ms. Trace:
I20260709 13:15:28.118691 11469 rpcz_store.cc:276] 0709 13:15:26.682749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.683624 (+   875us) service_pool.cc:224] Handling call
0709 13:15:26.691093 (+  7469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:26.719184 (+ 28091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:26.719214 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:26.719703 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:15:26.861805 (+142102us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:26.861844 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.861925 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.117897 (+1255972us) raft_consensus.cc:1685] finished
0709 13:15:28.117916 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.118117 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27327,"raft.queue_time_us":31133,"spinlock_wait_cycles":5844352,"thread_start_us":141752,"threads_started":1,"wal-append.queue_time_us":502467}
I20260709 13:15:28.163986 12812 raft_consensus.cc:993] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:28.164570 12812 raft_consensus.cc:1081] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:15:28.170027 12769 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.170614 12769 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.171257 12795 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.171797 12795 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.172724 12769 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.173255 12769 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.173805 12795 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.173658 12769 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.174364 12795 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.175287 12698 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.175774 12698 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.176046 12769 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.177778 12698 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.177920 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1257) took 1972 ms. Trace:
I20260709 13:15:28.178203 11471 rpcz_store.cc:276] 0709 13:15:26.205122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.205232 (+   110us) service_pool.cc:224] Handling call
0709 13:15:26.247852 (+ 42620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.619389 (+371537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.177738 (+1558349us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54002a4754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:28.177873 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833990400}
I20260709 13:15:28.174835 12795 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.181962 12795 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.184089 12769 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.184413 12712 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.184587 12769 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.184883 12712 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:28.186414 12084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.008s
I20260709 13:15:28.186604 12769 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.186686 12712 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:28.186811 12084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.001s	sys 0.009s
I20260709 13:15:28.187306 12769 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.187340 12712 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.187686 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.188035 12712 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.184602 12698 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.190644 12712 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.190881 12698 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.191464 12769 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.192912 12698 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.193782 12769 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.194296 12769 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:28.194770 12086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.010s	sys 0.002s
I20260709 13:15:28.194744 12769 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.183841 12858 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.195900 12858 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.196653 12769 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.198143 12858 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:28.195497 12086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.011s	sys 0.002s
I20260709 13:15:28.199736 12733 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.200217 12733 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.202162 12733 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.211236 12801 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:28.211917 12801 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.213254 12822 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:28.214692 12801 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.216045 12802 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:28.217475 12801 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.221622 12801 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:28.222092 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.222402 12801 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.222956 12786 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:28.223060 12812 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.223469 12786 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:28.223764 12812 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.223476 12801 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.224457 12786 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.226101 12766 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.224229 12812 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:28.226585 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.226676 12766 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.228098 12837 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.228528 12837 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.228950 12766 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.229445 12766 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.229863 12766 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.230497 12837 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.230976 12837 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.231415 12837 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.232642 12857 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.233274 12857 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.234364 12812 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.236491 12795 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.236915 12858 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.237004 12795 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.237489 12858 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.239166 12859 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.239437 12795 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.239981 12795 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.239813 12859 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.240448 12795 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.242420 12859 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.242782 12698 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.245421 12698 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.246122 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:28.246803 11396 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:28.247300 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1406) took 1381 ms (client timeout 1545 ms). Trace:
I20260709 13:15:28.247371 12698 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:28.247673 11396 rpcz_store.cc:269] 0709 13:15:26.866295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.867135 (+   840us) service_pool.cc:224] Handling call
0709 13:15:28.245961 (+1378826us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:28.247224 (+  1263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2780416}
I20260709 13:15:28.242470 12795 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.246613 12858 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.249651 12812 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.250627 12861 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:28.253234 12861 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.261754 12861 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.263245 12762 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.263970 12762 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.266359 12762 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.267251 12762 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.262224 12861 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.267774 12762 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.268894 12861 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.271307 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:28.271827 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.272639 12861 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.273464 12733 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.273681 11548 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:28.274013 12733 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.275058 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.275776 11548 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:28.276141 12733 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.276803 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:28.281987 11550 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.283942 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.284407 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.284054 11548 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.290614 11550 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:28.291678 11471 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:28.291736 11543 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.293121 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:28.285521 12801 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.283277 12766 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.295855 12801 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.300020 12762 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.294267 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.306289 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:28.305487 12837 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.307060 12837 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:28.309659 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.310092 12801 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.305058 12766 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.310904 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.310612 12801 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:28.311388 11470 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 795) took 2117 ms (client timeout 1987 ms). Trace:
I20260709 13:15:28.312623 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.312273 12812 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:28.309661 12860 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:28.312557 11470 rpcz_store.cc:269] 0709 13:15:26.193692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.198896 (+  5204us) service_pool.cc:224] Handling call
0709 13:15:28.311169 (+2112273us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400213254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:28.311288 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890212608}
I20260709 13:15:28.314481 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:28.314795 11474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.559s	user 0.021s	sys 0.030s
W20260709 13:15:28.315210 11474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.559s	user 0.022s	sys 0.030s
I20260709 13:15:28.315106 11551 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.315912 12822 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:28.316066 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 66) took 1687 ms. Trace:
I20260709 13:15:28.311457 12766 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.316615 12861 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:28.301805 11547 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.317082 12733 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.317417 11474 rpcz_store.cc:276] 0709 13:15:26.629045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.631830 (+  2785us) service_pool.cc:224] Handling call
0709 13:15:26.639488 (+  7658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:26.718930 (+ 79442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:26.718961 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:26.755454 (+ 36493us) log.cc:844] Serialized 26 byte log entry
0709 13:15:28.315687 (+1560233us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:28.315733 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.315825 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.315869 (+    44us) raft_consensus.cc:1685] finished
0709 13:15:28.315885 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.316015 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77297,"raft.queue_time_us":77871,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":6132,"thread_start_us":1559717,"threads_started":1,"wal-append.queue_time_us":1221138}
I20260709 13:15:28.320264 12733 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.320127 12861 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.323163 12733 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.326608 12766 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.319597 12822 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:28.327977 12822 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:28.328677 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.329519 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.331051 12860 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.331490 12860 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.332855 12860 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.335487 12768 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:28.348399 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 1.43281e+06us
I20260709 13:15:28.343272 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.349491 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.350577 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:28.350894 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.351672 12762 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.352005 12866 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.351843 12813 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.352797 12866 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.353500 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:28.353379 12866 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.352818 12813 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:28.354633 12813 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.349547 12812 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:28.356824 12857 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.356627 12812 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:28.360790 11442 leader_election.cc:336] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.501s (SENT)
I20260709 13:15:28.361227 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:28.361375 12860 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:28.362215 11442 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.504s (SENT)
I20260709 13:15:28.362377 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:28.362612 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:28.364780 12812 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.364969 12801 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.365756 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:28.366536 11477 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:28.366743 12860 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.368078 11394 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:15:28.368587 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1405) took 1502 ms (client timeout 1533 ms). Trace:
I20260709 13:15:28.368657 11551 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:28.368942 11394 rpcz_store.cc:269] 0709 13:15:26.866030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.871002 (+  4972us) service_pool.cc:224] Handling call
0709 13:15:28.368531 (+1497529us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:28.366911 11477 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1255) took 2163 ms (client timeout 1527 ms). Trace:
I20260709 13:15:28.370596 12733 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.371187 12733 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.373021 12733 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:28.373497 11477 rpcz_store.cc:269] 0709 13:15:26.203859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.203967 (+   108us) service_pool.cc:224] Handling call
0709 13:15:28.365608 (+2161641us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:28.366849 (+  1241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9136896}
I20260709 13:15:28.374533 12862 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:28.375160 12862 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:28.375614 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:28.375988 12862 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.387187 12813 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:28.388222 11365 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.527s (SENT) [suppressed 47 similar messages]
I20260709 13:15:28.374481 12762 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:28.380959 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.390719 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.392067 12813 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.389175 12801 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.390461 12857 raft_consensus.cc:2764] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:28.390136 12812 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.390892 12768 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.401461 12812 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.404302 12812 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.393117 12687 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.401639 12768 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.406944 12866 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.794108 12866 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.413322 12857 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.444016 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:28.444684 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:28.795406 12857 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.797501 12857 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:28.799217 12861 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.799404 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:28.444793 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:28.800153 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.800760 12117 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:28.452751 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:28.456243 12858 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.801937 11395 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.802301 12858 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.803028 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:28.803624 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:28.804387 12858 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.457772 12863 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.804590 12768 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.805047 12863 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:28.459352 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 71157.3us
I20260709 13:15:28.807554 11393 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:28.808501 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:28.460258 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 69604.8us
I20260709 13:15:28.809017 11297 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.471740 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:28.468905 11547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1511) took 2009 ms (client timeout 1886 ms). Trace:
I20260709 13:15:28.809890 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:28.810344 11477 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
W20260709 13:15:28.810294 11547 rpcz_store.cc:269] 0709 13:15:26.459800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.461759 (+  1959us) service_pool.cc:224] Handling call
0709 13:15:28.468727 (+2006968us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003ea8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:28.468838 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4004424960}
I20260709 13:15:28.472500 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.811082 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.811714 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:28.811793 12857 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.811964 12768 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.812179 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.474954 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:28.812538 12857 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.812456 12768 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.472877 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.473229 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.813488 11471 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:28.475656 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.480345 12874 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.486737 12855 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:28.480427 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.483011 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.814739 12855 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.508927 12854 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:28.509485 12697 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.516798 12766 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.524302 12839 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.529152 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.530486 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.539397 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:28.825911 12855 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.826027 11551 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.826321 12786 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.540179 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.540841 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.542269 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.542936 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.827975 11543 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:28.828946 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.829028 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.829501 11543 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:28.830265 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.544051 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.830349 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:28.546689 12864 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.548059 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.830978 11546 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:28.831496 11548 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:28.536384 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 865) took 1671 ms. Trace:
I20260709 13:15:28.640956 12769 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.643366 12795 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.644116 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.636117 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.837437 12804 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.833835 12864 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:28.657605 12883 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.838467 12864 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.839058 12766 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.663652 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.839609 12864 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.822446 12732 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.840359 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.840997 11549 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:28.840852 12732 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.841930 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.670761 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.671789 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:28.679605 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.821861 12704 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.836519 12777 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.685096 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.827921 12857 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.400856 12801 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.823899 12687 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.734277 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.123s	user 0.003s	sys 0.006s
I20260709 13:15:28.736807 12886 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.700376 12712 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.702371 12778 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.688066 12884 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:28.749120 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.851789 12118 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:28.751051 11300 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 139) took 3944 ms. Trace:
I20260709 13:15:28.754170 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 863) took 1889 ms. Trace:
I20260709 13:15:28.758117 12757 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.767999 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.856967 12113 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:28.856690 12712 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.771744 12816 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.857756 12816 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:28.844259 12704 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.858560 12816 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.859050 12778 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.859589 12816 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.860026 12704 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:28.860558 12704 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.861117 12778 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.860989 12704 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.843030 12769 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.863214 12768 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:28.775637 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.777391 12887 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:28.780167 12698 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.783974 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.784430 12812 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:28.784883 12802 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.786309 12681 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:28.447415 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:28.803711 11395 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:28.461653 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 63004.5us
I20260709 13:15:28.812309 11547 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.812903 11474 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.491880 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:28.399328 12762 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.494693 12115 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:28.815979 11398 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:28.818111 12123 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:28.815639 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:28.820909 12858 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.814651 12874 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.546332 12859 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.832074 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:28.834292 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:28.689944 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.843798 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:28.845528 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.004s	sys 0.007s
I20260709 13:15:28.847008 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:28.802315 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.855147 11551 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:28.823601 12775 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.842892 12732 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.862437 12886 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.047214 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:28.836025 11400 rpcz_store.cc:276] 0709 13:15:26.864961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.867324 (+  2363us) service_pool.cc:224] Handling call
0709 13:15:28.535526 (+1668202us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4eee 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:28.535725 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.535940 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.535976 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.536069 (+    93us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:28.536111 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.536179 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.536311 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6122368}
I20260709 13:15:28.864459 12865 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:28.864120 12884 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.844726 12857 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.858871 12712 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:28.954694 12698 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.844357 12777 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.882095 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:28.839263 12883 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:29.058815 12775 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.986043 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:29.059100 12802 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.121493 12802 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.121577 11474 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:28.845484 12862 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.122412 12862 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.844550 12855 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:29.061069 12886 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.123240 12855 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:28.872052 11393 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:28.843176 12837 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.954914 12887 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.063488 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:28.845296 12687 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:28.862866 12704 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.065352 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:28.852746 11300 rpcz_store.cc:276] 0709 13:15:24.806515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.806686 (+   171us) service_pool.cc:224] Handling call
0709 13:15:28.750939 (+3944253us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:25.396637 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:25.397428 (+   791us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:25.397471 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:15:25.397500 (+    29us) tablet.cc:662] Decoding operations
0709 13:15:25.397939 (+   439us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:25.398040 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:25.398068 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:25.398358 (+   290us) tablet.cc:701] Row locks acquired
0709 13:15:25.408543 (+ 10185us) write_op.cc:260] PREPARE: finished
0709 13:15:25.426828 (+ 18285us) write_op.cc:270] Start()
0709 13:15:25.483149 (+ 56321us) write_op.cc:276] Timestamp: P: 1783602925421661 usec, L: 0
0709 13:15:25.483200 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:15:25.484002 (+   802us) log.cc:844] Serialized 587 byte log entry
0709 13:15:25.866573 (+382571us) op_driver.cc:464] REPLICATION: finished
0709 13:15:26.250607 (+384034us) write_op.cc:301] APPLY: starting
0709 13:15:26.250721 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:26.250830 (+   109us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:26.296584 (+ 45754us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:26.296896 (+   312us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:26.297154 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:15:26.297223 (+    69us) write_op.cc:312] APPLY: finished
0709 13:15:26.297454 (+   231us) log.cc:844] Serialized 23 byte log entry
0709 13:15:26.297641 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.297722 (+    81us) write_op.cc:454] Released schema lock
0709 13:15:26.297772 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:26.298153 (+   381us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.298181 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":427917,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":302808,"apply.run_cpu_time_us":2413,"apply.run_wall_time_us":93162,"num_ops":1,"prepare.queue_time_us":207805,"prepare.run_cpu_time_us":22676,"prepare.run_wall_time_us":139708,"raft.queue_time_us":196617,"raft.run_cpu_time_us":43953,"raft.run_wall_time_us":242705,"replication_time_us":383217,"thread_start_us":602652,"threads_started":4,"wal-append.queue_time_us":124593,"wal-append.run_cpu_time_us":22036,"wal-append.run_wall_time_us":1718475}]]}
I20260709 13:15:29.066138 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.056495 12119 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.066959 12858 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.067893 12865 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:29.079185 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 232266us
I20260709 13:15:29.068048 12764 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:29.080179 12769 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.080825 12863 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.081146 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:29.059017 11442 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.533s (SENT)
I20260709 13:15:29.081702 12794 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:29.085000 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.087158 11402 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.087920 11546 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.088676 11545 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.048266 11298 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 227) took 3752 ms. Trace:
I20260709 13:15:29.048718 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.843256 12795 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.056443 12801 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.087180 12812 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.107223 11550 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.132364 11470 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:29.152221 11548 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:28.874907 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.843655 12112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 304) took 2527 ms (client timeout 1573 ms). Trace:
I20260709 13:15:29.119143 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.090790 12861 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.035120 12867 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.975952 12866 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:29.128707 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.217749 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.257567 11398 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.256886 12861 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.258437 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.257900 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.126950 11543 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.130950 11396 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:29.133869 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:29.259212 11398 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.259616 12861 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.260111 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.257207 12867 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.260783 11398 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.260953 12812 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.261128 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:28.855208 11401 rpcz_store.cc:276] 0709 13:15:26.864418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.867590 (+  3172us) service_pool.cc:224] Handling call
0709 13:15:28.753281 (+1885691us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:28.753510 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.753724 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.753765 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.753825 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:28.753867 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.753965 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.754103 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8527616}
I20260709 13:15:29.261720 12118 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.262209 12867 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.262411 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.140055 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.262784 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:29.263067 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:29.263465 11473 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
W20260709 13:15:29.263866 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 204892us
I20260709 13:15:29.145448 11397 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.130357 12862 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:29.097653 12859 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.129842 12681 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.264528 11298 rpcz_store.cc:276] 0709 13:15:25.098314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.098451 (+   137us) service_pool.cc:224] Handling call
0709 13:15:28.851133 (+3752682us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2163663,"reported_tablets":6}
I20260709 13:15:29.265242 11401 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.266047 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.266239 12859 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.134684 12855 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:29.255535 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 269592us
I20260709 13:15:29.264730 12862 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:29.267593 12862 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.267905 12861 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:29.269296 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.270180 12769 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.143971 11547 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:29.129722 11549 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:29.136821 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.258533 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.264151 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.266394 12681 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.143720 11400 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.151829 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.270076 11441 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:28.976464 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.270522 11401 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:29.301370 12112 rpcz_store.cc:269] 0709 13:15:26.316239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.317614 (+  1375us) service_pool.cc:224] Handling call
0709 13:15:28.671668 (+2354054us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4eee 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:28.843558 (+171890us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32287488}
I20260709 13:15:29.302099 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.280949 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.271941 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:29.257413 12866 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:29.306216 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 172457us
I20260709 13:15:29.157148 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:29.310810 12764 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.302774 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.316548 12764 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:29.311857 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.274843 11398 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.318406 12858 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.318573 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.319136 11398 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.319895 12887 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.305388 12866 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.158681 12119 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:29.322754 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:29.321002 12716 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:15:29.326367 11546 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.326346 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.327696 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.322675 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.322285 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:29.329026 11551 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.329454 11545 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.330382 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.302273 12769 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:29.272534 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 454511us
W20260709 13:15:29.344808 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 73717.4us
I20260709 13:15:29.303637 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.257362 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.347137 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.347846 11395 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:29.348974 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.349690 11395 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.350750 11398 raft_consensus.cc:2410] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:15:29.354641 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:29.334264 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 196803us
I20260709 13:15:29.355329 11398 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 2: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.357348 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:29.331463 12123 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:29.304780 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:29.323410 12819 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.335777 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.364820 11551 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
W20260709 13:15:29.364881 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 60184.5us
I20260709 13:15:29.364537 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:29.278475 11442 leader_election.cc:336] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.545s (SENT)
I20260709 13:15:29.302850 11401 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
W20260709 13:15:29.366879 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 88431.9us
W20260709 13:15:29.333575 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 181552us
I20260709 13:15:29.368309 11367 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.302857 12795 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.357973 11473 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.335035 12119 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.350659 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.375418 11477 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.369240 11367 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.376279 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:29.241843 12116 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:29.351908 12769 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.370929 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.386806 11551 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.303670 12891 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.390429 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.351402 12858 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:29.377264 11468 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:29.377270 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.362452 11545 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.370163 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.205394 12704 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.173s
I20260709 13:15:29.344218 12839 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.344388 12887 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:29.384335 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.351603 12681 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.343787 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.357996 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.414537 11398 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:29.415616 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.416350 11398 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
W20260709 13:15:29.416394 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44726.2us
I20260709 13:15:29.366880 12861 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.368719 12859 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:15:29.394081 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:29.394263 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.435974 11551 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.436179 11475 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:29.381382 11442 leader_election.cc:336] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.574s (SENT)
I20260709 13:15:29.436820 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:29.396095 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.371690 12819 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.310810 12865 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.369112 12894 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.444512 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.407333 12862 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.445149 11472 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.446053 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.446607 11472 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.404727 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.447072 12866 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.447810 11547 raft_consensus.cc:2410] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:29.404338 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.448158 12837 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:29.448594 11393 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.449152 12837 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:29.449503 12837 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:29.450278 12837 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.395099 12867 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.451774 12867 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:29.452602 12867 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:29.453845 12786 consensus_meta.cc:287] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.000s
I20260709 13:15:29.454458 11544 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.440512 12887 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:29.435372 12861 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.405110 12795 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.408685 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.407229 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.405412 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.408634 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.461217 11550 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:29.461041 11477 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.380970 12886 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:29.439961 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.420359 11473 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:29.374833 12883 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.385572 12712 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:29.425625 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.426188 11395 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.438619 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:29.408331 12144 debug-util.cc:398] Leaking SignalData structure 0x7b0800307060 after lost signal to thread 10255
W20260709 13:15:29.404752 12816 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.002s
W20260709 13:15:29.405181 12864 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:15:29.475170 12883 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.475620 12883 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.473165 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.476033 12883 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.476567 11398 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:29.478499 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:29.441823 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 60489us
W20260709 13:15:29.412722 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.394147 12855 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:29.442977 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 59614.5us
W20260709 13:15:29.415669 12697 consensus_meta.cc:287] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.000s
I20260709 13:15:29.456811 12118 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:29.457052 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61043.4us
I20260709 13:15:29.406224 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.461126 11468 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:29.419579 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.465030 11551 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:29.465735 11546 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:29.423424 12681 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.467967 11545 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.439296 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:29.469099 11394 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.604s	user 0.005s	sys 0.005s
I20260709 13:15:29.411201 12858 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.455871 12887 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.734736 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:29.664584 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.739419 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:29.508322 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:29.718276 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.738945 11442 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.604s (SENT)
I20260709 13:15:29.726516 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.744889 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.460042 11469 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.460721 12795 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:29.508510 12757 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.001s
I20260709 13:15:29.743659 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:29.755368 12795 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.762385 12777 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.763146 12786 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.751732 12831 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:29.763552 12786 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.745862 12704 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.764122 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 74) took 1292 ms. Trace:
I20260709 13:15:29.764649 11470 rpcz_store.cc:276] 0709 13:15:28.471787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.475398 (+  3611us) service_pool.cc:224] Handling call
0709 13:15:29.763894 (+1288496us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400062a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:29.764060 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716109056}
I20260709 13:15:29.764691 12704 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.739868 11548 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.745059 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.740724 11475 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:29.741410 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:29.740731 11471 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:29.459128 12857 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.744647 11395 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:29.781041 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.464483 12886 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.469756 12712 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.463227 12862 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.480355 12839 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.791599 12712 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.764000 12786 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.772518 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:29.747035 12837 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.773761 12775 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.709836 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1246) took 1171 ms. Trace:
I20260709 13:15:29.483240 12698 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.775017 12785 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.775270 12757 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.483491 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.775492 11468 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:29.484881 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:29.760097 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:29.815860 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 55855.1us
I20260709 13:15:29.815973 12732 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.816431 12732 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.816576 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:29.817564 12762 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.817931 12762 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.818254 12762 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.758621 12813 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.784027 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.785440 12764 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.781232 12777 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.787752 12795 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.811389 12757 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.773458 12704 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:29.782048 12144 debug-util.cc:398] Leaking SignalData structure 0x7b0800307740 after lost signal to thread 11334
I20260709 13:15:29.757445 12861 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.769138 12855 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:29.790038 12886 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.816812 12732 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.763921 12831 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.827735 12757 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.813160 12698 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.814761 12785 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.668607 12864 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.803114 12902 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:29.812388 12862 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.822620 12813 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.838402 12902 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:29.838657 12858 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.833652 12769 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.839596 12902 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.841238 12769 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.834414 12831 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.841792 12862 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.846978 12862 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:29.854616 12806 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.854890 12795 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.855173 12806 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:29.856107 12806 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.857081 12806 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.857883 12813 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:29.842686 12864 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.859267 12855 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:29.917876 12883 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.925151 12883 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:29.921896 12911 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:29.920325 12886 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:29.930955 12886 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:29.955264 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.783423 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.962097 11395 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.742132 11394 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:29.964787 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.966629 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 872) took 1146 ms. Trace:
I20260709 13:15:29.968007 11395 rpcz_store.cc:276] 0709 13:15:28.820264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.964052 (+1143788us) service_pool.cc:224] Handling call
0709 13:15:29.966503 (+  2451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.970216 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.788169 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1414) took 1344 ms. Trace:
I20260709 13:15:29.775013 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:29.989378 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.990612 11401 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.972302 11398 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:29.973280 11399 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:15:29.709952 11400 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:15:29.992631 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1425) took 1544 ms (client timeout 1573 ms). Trace:
W20260709 13:15:29.993108 11400 rpcz_store.cc:269] 0709 13:15:28.448136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.063216 (+615080us) service_pool.cc:224] Handling call
0709 13:15:29.992432 (+929216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.994544 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:29.788012 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.803563 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.743945 11442 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.637s (SENT)
I20260709 13:15:29.974166 11402 raft_consensus.cc:2368] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:29.986141 11396 rpcz_store.cc:276] 0709 13:15:28.443950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.444553 (+   603us) service_pool.cc:224] Handling call
0709 13:15:29.787980 (+1343427us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:29.788114 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759558016}
I20260709 13:15:29.814703 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.776732 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.072717 11472 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:29.814059 11474 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:29.787302 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:29.809746 11545 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.081609 11477 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:29.822774 11548 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.083385 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.084569 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.084738 11545 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:29.809216 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:29.810711 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.042135 11400 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:30.090374 12864 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.042277 12681 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:30.091130 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.091881 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.805759 11542 rpcz_store.cc:276] 0709 13:15:28.538288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.843422 (+305134us) service_pool.cc:224] Handling call
0709 13:15:28.843733 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.148997 (+305264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.157005 (+  8008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.197882 (+ 40877us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:29.218902 (+ 21020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.709484 (+490582us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54002c56d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:29.709554 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.709750 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670985600}
I20260709 13:15:29.836994 12698 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:30.064330 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:29.820029 11471 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.068522 11468 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:30.045158 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.078456 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.107529 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 1.
I20260709 13:15:30.107044 12698 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:30.108673 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 364718us
I20260709 13:15:30.109166 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:30.109850 11442 leader_election.cc:336] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.645s (SENT)
I20260709 13:15:30.110191 11469 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.110466 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.105294 11401 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.097424 12786 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.113536 12819 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.114066 12786 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.074815 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.115608 11472 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.116190 12866 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:30.099481 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080037a840 after lost signal to thread 11426
W20260709 13:15:30.118572 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1460 after lost signal to thread 11501
I20260709 13:15:30.100010 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:30.118896 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1200 after lost signal to thread 11502
I20260709 13:15:30.119465 11400 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
W20260709 13:15:30.120554 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080030d0c0 after lost signal to thread 11575
I20260709 13:15:30.120795 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.121712 11400 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
W20260709 13:15:30.122251 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080037e700 after lost signal to thread 12147
I20260709 13:15:30.100479 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.116822 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.123548 11474 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:30.124061 11472 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.125312 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.125696 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.100745 11548 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.101037 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.128409 11396 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:30.129055 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.129555 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.102008 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.104417 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.104671 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.122542 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.130960 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.134049 11396 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:30.134157 12785 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.134606 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.145972 11542 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:30.146133 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1413) took 3257 ms (client timeout 1921 ms). Trace:
I20260709 13:15:30.147730 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:30.148420 11442 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.652s (SENT)
I20260709 13:15:30.149057 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.150159 12866 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.151103 12866 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.151182 11474 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:30.152072 11393 consensus_meta.cc:287] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.637s	user 0.000s	sys 0.022s
I20260709 13:15:30.152778 11393 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:30.135581 12764 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.131592 11550 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:30.150377 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.154282 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.154383 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.133277 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.155997 11551 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:30.131837 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.136547 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.158398 11545 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.132122 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:30.160214 12762 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:30.153059 11442 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.652s (SENT)
I20260709 13:15:30.132627 11471 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:30.171347 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 426348us
W20260709 13:15:30.171845 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 88) took 1715 ms (client timeout 2120 ms). Trace:
I20260709 13:15:30.172488 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:30.172348 11402 rpcz_store.cc:269] 0709 13:15:28.456539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.310823 (+854284us) service_pool.cc:224] Handling call
0709 13:15:30.170621 (+859798us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54003a75d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.171784 (+  1163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507935488}
I20260709 13:15:30.173491 12785 raft_consensus.cc:2764] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:30.173882 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.174881 11402 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:30.176746 11442 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.704s (SENT)
I20260709 13:15:30.176834 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.177416 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.178359 12864 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.178624 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.179229 12757 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.181432 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 82) took 1043 ms. Trace:
I20260709 13:15:30.181250 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.133018 12832 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.182400 12832 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.182955 12832 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:30.183282 11543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1247) took 1644 ms (client timeout 2145 ms). Trace:
W20260709 13:15:30.183705 11543 rpcz_store.cc:269] 0709 13:15:28.538515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.830066 (+291551us) service_pool.cc:224] Handling call
0709 13:15:30.182920 (+1352854us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54000b2f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.183100 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681126272}
I20260709 13:15:30.183266 12831 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.174661 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.182585 12732 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:30.132620 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.187595 11546 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:30.188201 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.188717 12732 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:30.181941 11476 rpcz_store.cc:276] 0709 13:15:29.137610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.141137 (+  3527us) service_pool.cc:224] Handling call
0709 13:15:30.180106 (+1038969us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:30.180455 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.180862 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.180902 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.180981 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.181034 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.181100 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.181230 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48630656}
I20260709 13:15:30.189715 12785 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.190228 12785 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.133085 11477 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.190583 12785 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.144538 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.135335 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.192608 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.193637 11396 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.194658 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:30.195444 11396 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:30.132277 12085 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:30.122817 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.164382 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.126782 11472 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.163035 12712 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.207320 12112 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:30.208241 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 76373.5us
I20260709 13:15:30.208763 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.209594 11396 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.208243 12766 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.210429 12766 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.210871 12766 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.209245 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:30.169706 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 873) took 1349 ms. Trace:
I20260709 13:15:30.210767 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.213801 11472 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.217733 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.218708 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.218847 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.219336 12831 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.219555 12116 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.219722 12113 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:30.220965 12114 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:30.221855 11544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1241) took 1684 ms (client timeout 2104 ms). Trace:
I20260709 13:15:30.221801 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.222285 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:30.222365 11544 rpcz_store.cc:269] 0709 13:15:28.537079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.539929 (+  2850us) service_pool.cc:224] Handling call
0709 13:15:30.221483 (+1681554us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400269154. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.221690 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442103040}
I20260709 13:15:30.223197 12118 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.224469 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:30.226217 12687 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:30.227708 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1417) took 1782 ms (client timeout 1845 ms). Trace:
W20260709 13:15:30.228171 11397 rpcz_store.cc:269] 0709 13:15:28.444856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.447127 (+  2271us) service_pool.cc:224] Handling call
0709 13:15:30.227288 (+1780161us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b540019a854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.227518 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67018112}
I20260709 13:15:30.228883 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.229887 12804 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.230787 12804 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.231217 12804 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.231562 12804 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.232298 11397 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.139104 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.245715 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1424) took 1798 ms. Trace:
I20260709 13:15:30.201799 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.093487 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:30.204442 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.212091 11398 rpcz_store.cc:276] 0709 13:15:28.820561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.477639 (+657078us) service_pool.cc:224] Handling call
0709 13:15:30.169465 (+691826us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400349754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.169648 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533378304}
W20260709 13:15:30.205703 11442 leader_election.cc:336] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.845s (SENT)
I20260709 13:15:30.205621 11400 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
W20260709 13:15:30.207253 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62920.9us
I20260709 13:15:30.257612 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.233330 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 344) took 1169 ms. Trace:
I20260709 13:15:30.234253 11393 raft_consensus.cc:2368] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4.
I20260709 13:15:30.235913 12712 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.236850 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.237572 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1251) took 1692 ms. Trace:
I20260709 13:15:30.238503 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.181782 12757 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:30.146651 11394 rpcz_store.cc:269] 0709 13:15:28.443573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.443704 (+   131us) service_pool.cc:224] Handling call
0709 13:15:30.145787 (+1702083us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b54003d36d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.145954 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154929536}
I20260709 13:15:30.239696 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.240473 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.241181 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.241314 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.241693 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.246459 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:30.214462 12762 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:30.247290 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.249341 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.249017 12832 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.250643 11397 rpcz_store.cc:276] 0709 13:15:28.446870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.229164 (+1782294us) service_pool.cc:224] Handling call
0709 13:15:30.231571 (+  2407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.245415 (+ 13844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.245638 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.161028 11469 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:30.256517 12118 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:30.232933 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:30.257578 11396 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:30.208284 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 345) took 1144 ms. Trace:
I20260709 13:15:30.261950 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:30.262785 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 63437.5us
I20260709 13:15:30.211941 12785 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.262825 11549 rpcz_store.cc:276] 0709 13:15:28.545156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.841671 (+296515us) service_pool.cc:224] Handling call
0709 13:15:30.237242 (+1395571us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001ec8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.237391 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660429056}
I20260709 13:15:30.272861 11402 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:30.304836 11468 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.313351 11547 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.314181 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.315461 11477 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.313359 12113 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:30.259402 12114 rpcz_store.cc:276] 0709 13:15:29.063823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.328531 (+264708us) service_pool.cc:224] Handling call
0709 13:15:30.233078 (+904547us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54001d0bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.233243 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832349952}
W20260709 13:15:30.318224 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48999.1us
I20260709 13:15:30.275154 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.336879 11401 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 3. Ops: []
W20260709 13:15:30.326800 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52436.2us
W20260709 13:15:30.337841 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.338320 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.338661 12839 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.338753 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.339670 12863 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:30.340282 12832 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.340402 11402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:15:30.340476 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:30.341169 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.341650 12716 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.274168 12764 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:15:30.342604 12716 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:30.343076 12716 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:30.343381 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.343573 12716 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:30.347373 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.347970 11395 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.349133 12831 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:30.349679 12115 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:30.351205 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:30.351877 11397 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.355013 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.355588 12766 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.356185 12819 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:30.357182 12698 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:30.357378 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1403) took 3491 ms (client timeout 1501 ms). Trace:
I20260709 13:15:30.357641 12698 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.356503 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.358278 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.358539 11469 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.346578 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.347405 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.341265 11402 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:30.349059 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:30.348033 12862 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.361073 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1432) took 1910 ms. Trace:
I20260709 13:15:30.362124 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1429) took 1912 ms. Trace:
W20260709 13:15:30.357884 11399 rpcz_store.cc:269] 0709 13:15:26.865499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.865623 (+   124us) service_pool.cc:224] Handling call
0709 13:15:30.357139 (+3491516us) spinlock_profiling.cc:243] Waited 3.1 s on lock 0x7b5400349254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.357290 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7118833024}
I20260709 13:15:30.363479 11402 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.363517 11398 rpcz_store.cc:276] 0709 13:15:28.449376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.264110 (+1814734us) service_pool.cc:224] Handling call
0709 13:15:30.305335 (+ 41225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.361807 (+ 56472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.361835 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.361883 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:30.361914 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.361976 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.362085 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.364705 12117 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.365403 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1448) took 1018 ms. Trace:
I20260709 13:15:30.358064 11400 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.365888 12785 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.365834 11393 rpcz_store.cc:276] 0709 13:15:29.346539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.153887 (+807348us) service_pool.cc:224] Handling call
0709 13:15:30.365173 (+211286us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400199e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:30.365340 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182053632}
I20260709 13:15:30.363427 11401 rpcz_store.cc:276] 0709 13:15:28.450249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.273977 (+1823728us) service_pool.cc:224] Handling call
0709 13:15:30.336695 (+ 62718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.337754 (+  1059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.360973 (+ 23219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.358141 12712 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.381305 12712 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.381783 12712 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.366818 12819 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.379081 12687 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.358083 12698 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.371806 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.375912 11474 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.379200 12893 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.379300 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:30.318893 11470 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.271492 12112 rpcz_store.cc:276] 0709 13:15:29.064063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.205502 (+1141439us) service_pool.cc:224] Handling call
0709 13:15:30.206091 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.207927 (+  1836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.208209 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:30.390899 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44542.2us
I20260709 13:15:30.392205 11397 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.353472 12716 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.366428 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1430) took 1916 ms. Trace:
I20260709 13:15:30.357880 12766 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:30.361713 12863 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:30.363281 12862 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.396219 11468 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:30.395757 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.394017 11477 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.396519 11395 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.317250 11476 raft_consensus.cc:2376] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3. Re-sending same reply.
I20260709 13:15:30.391908 12687 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:30.394308 12893 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.391309 12806 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:30.407399 11473 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:30.404441 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 198867us
I20260709 13:15:30.410344 11475 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:30.411919 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53804.4us
I20260709 13:15:30.405711 12862 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:30.415571 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46934.5us
I20260709 13:15:30.418794 12757 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.419256 12757 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.419595 12874 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.400554 11400 rpcz_store.cc:276] 0709 13:15:28.449679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.269380 (+1819701us) service_pool.cc:224] Handling call
0709 13:15:30.316668 (+ 47288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.366172 (+ 49504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.366369 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.409627 12884 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.409925 12893 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.409902 12806 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:30.419632 12757 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.425235 12863 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.413633 12915 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:15:30.426229 12903 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.428225 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.431825 11469 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:30.436348 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 88950.7us
I20260709 13:15:30.410758 12804 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.440555 11397 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.433199 12806 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.442569 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 89) took 1984 ms. Trace:
I20260709 13:15:30.443030 11397 rpcz_store.cc:276] 0709 13:15:28.456893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.433640 (+1976747us) service_pool.cc:224] Handling call
0709 13:15:30.439421 (+  5781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.440874 (+  1453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.441026 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.444012 12894 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.444545 11394 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.467453 12894 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.467878 12894 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.467957 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 93) took 2009 ms. Trace:
I20260709 13:15:30.468386 11394 rpcz_store.cc:276] 0709 13:15:28.458201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.434713 (+1976512us) service_pool.cc:224] Handling call
0709 13:15:30.440354 (+  5641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.467642 (+ 27288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.467874 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.467454 12116 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:30.470495 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 92) took 2012 ms. Trace:
I20260709 13:15:30.471387 11393 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.436154 12903 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.472066 12903 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.472203 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 874) took 1651 ms. Trace:
I20260709 13:15:30.472647 11393 rpcz_store.cc:276] 0709 13:15:28.820790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.444127 (+1623337us) service_pool.cc:224] Handling call
0709 13:15:30.465372 (+ 21245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.471948 (+  6576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.472145 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.479707 12712 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.482085 12712 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.484555 12903 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.462914 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:30.490769 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.441524 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.492966 11402 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.485210 11297 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 213) took 5157 ms. Trace:
W20260709 13:15:30.487227 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 71696.7us
I20260709 13:15:30.494886 11399 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:30.490332 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.489991 12757 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:30.491252 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.465116 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.496919 12819 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:30.497164 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.419616 12687 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.494652 11297 rpcz_store.cc:276] 0709 13:15:25.327326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.327529 (+   203us) service_pool.cc:224] Handling call
0709 13:15:30.484976 (+5157447us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:29.153708 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:29.324175 (+170467us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:29.324221 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:15:29.324247 (+    26us) tablet.cc:662] Decoding operations
0709 13:15:29.324630 (+   383us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:29.324719 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:29.324763 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:29.324986 (+   223us) tablet.cc:701] Row locks acquired
0709 13:15:29.325010 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:15:29.325175 (+   165us) write_op.cc:270] Start()
0709 13:15:29.325306 (+   131us) write_op.cc:276] Timestamp: P: 1783602929325125 usec, L: 0
0709 13:15:29.325333 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:15:29.405478 (+ 80145us) log.cc:844] Serialized 527 byte log entry
0709 13:15:29.843528 (+438050us) op_driver.cc:464] REPLICATION: finished
0709 13:15:29.927774 (+ 84246us) write_op.cc:301] APPLY: starting
0709 13:15:29.927869 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:29.927970 (+   101us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:29.928116 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:29.928334 (+   218us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:29.928511 (+   177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:15:29.928553 (+    42us) write_op.cc:312] APPLY: finished
0709 13:15:29.928777 (+   224us) log.cc:844] Serialized 23 byte log entry
0709 13:15:29.928940 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.929015 (+    75us) write_op.cc:454] Released schema lock
0709 13:15:29.929052 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:29.929357 (+   305us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.929385 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30916,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":84007,"apply.run_cpu_time_us":1573,"apply.run_wall_time_us":1572,"num_ops":1,"prepare.queue_time_us":314496,"prepare.run_cpu_time_us":13923,"prepare.run_wall_time_us":539700,"raft.queue_time_us":55037,"raft.run_cpu_time_us":82442,"raft.run_wall_time_us":84261,"replication_time_us":514004,"thread_start_us":672166,"threads_started":4,"wal-append.queue_time_us":381077}]]}
W20260709 13:15:30.494515 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.501667 11402 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.502018 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:30.502220 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41728 (request call id 1433) took 2051 ms. Trace:
I20260709 13:15:30.502871 11402 rpcz_store.cc:276] 0709 13:15:28.450602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.315413 (+1864811us) service_pool.cc:224] Handling call
0709 13:15:30.502114 (+186701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.470932 11400 rpcz_store.cc:276] 0709 13:15:28.457941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.432194 (+1974253us) service_pool.cc:224] Handling call
0709 13:15:30.432586 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.442597 (+ 10011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.455271 (+ 12674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.464293 (+  9022us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.465444 (+  1151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.470284 (+  4840us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.470432 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5755,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":144}
I20260709 13:15:30.504096 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1439) took 2051 ms. Trace:
I20260709 13:15:30.504570 11399 rpcz_store.cc:276] 0709 13:15:28.452386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.394745 (+1942359us) service_pool.cc:224] Handling call
0709 13:15:30.411132 (+ 16387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.495465 (+ 84333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.503835 (+  8370us) spinlock_profiling.cc:243] Waited 83.6 ms on lock 0x7b54003d4fd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:30.504025 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192176384}
I20260709 13:15:30.507576 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 101) took 1059 ms. Trace:
I20260709 13:15:30.508268 12716 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.504330 11401 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.513038 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 94) took 2054 ms. Trace:
I20260709 13:15:30.513408 11401 rpcz_store.cc:276] 0709 13:15:28.458491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.440248 (+1981757us) service_pool.cc:224] Handling call
0709 13:15:30.503936 (+ 63688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.512796 (+  8860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.512988 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.505589 11397 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:30.510649 11395 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:30.501693 11398 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:30.514859 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 869) took 1695 ms. Trace:
I20260709 13:15:30.514889 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 875) took 1693 ms. Trace:
I20260709 13:15:30.515487 11398 rpcz_store.cc:276] 0709 13:15:28.819361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.451099 (+1631738us) service_pool.cc:224] Handling call
0709 13:15:30.501061 (+ 49962us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:30.501303 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.514621 (+ 13318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.514811 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9623808}
I20260709 13:15:30.515558 11397 rpcz_store.cc:276] 0709 13:15:28.821026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.504819 (+1683793us) service_pool.cc:224] Handling call
0709 13:15:30.505178 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.514431 (+  9253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.514838 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:30.512002 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82746.4us
W20260709 13:15:30.515084 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51596.3us
I20260709 13:15:30.519381 11398 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:15:30.521229 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 83012.6us
I20260709 13:15:30.519634 12819 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:30.521955 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.511725 11394 rpcz_store.cc:276] 0709 13:15:29.448368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.506591 (+1058223us) service_pool.cc:224] Handling call
0709 13:15:30.506937 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.507162 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.507192 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.507250 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.507284 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.507351 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.507504 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.513873 12863 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.523048 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.523370 12863 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.523452 12831 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:30.524024 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:30.524577 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61764.9us
I20260709 13:15:30.523801 12863 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.522409 12819 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:30.517796 12712 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.532306 11401 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:30.536320 12764 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.536800 12874 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.537518 11365 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:30.543782 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 126609us
I20260709 13:15:30.543232 12764 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.545990 12915 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.546046 12764 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.546769 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.546900 12915 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:30.547431 12862 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.548338 12915 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.540542 12819 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.548656 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.550380 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.551187 12785 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.551580 12785 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.551918 12785 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.544402 12874 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.555243 12874 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.554615 12862 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.559391 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.560316 12884 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.560673 12884 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.531049 12757 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.542420 12712 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.578136 11397 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:30.576467 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.577131 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53189.9us
I20260709 13:15:30.579435 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:30.577806 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:30.580950 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 58829.4us
I20260709 13:15:30.560990 12884 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.571403 12915 raft_consensus.cc:2764] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:30.588395 12903 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.589399 12894 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.584336 12698 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.589402 12819 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.592936 12698 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.593679 12831 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.594061 12819 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.594624 12819 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.597963 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.598040 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.590075 11298 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.580741 12712 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.609601 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.620406 11471 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.595089 12894 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.609308 12903 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.609326 11298 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.627666 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:30.583539 12893 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.634801 11547 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.632637 11549 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:30.614908 12819 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.595837 12785 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:30.645710 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.645898 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:30.646622 12864 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:30.646816 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.647385 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.647547 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.649833 12874 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.651335 12687 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.651895 12832 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:30.653023 12786 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:30.631170 12903 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.624878 12862 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.704634 12915 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:30.704977 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:30.704686 12903 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.705615 12114 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.643071 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.659768 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.633795 12757 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.660240 12762 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:30.661713 11396 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 4. Ops: []
I20260709 13:15:30.662079 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.658689 12837 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:15:30.655529 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.662745 12893 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.643234 11298 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.660739 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.660712 12764 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.676223 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:30.714514 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 102183us
I20260709 13:15:30.685194 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:30.699476 11442 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.972s (SENT)
I20260709 13:15:30.643206 12819 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.628293 12804 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:30.706673 11544 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.719470 11545 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.719910 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.720340 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.721022 11545 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.721728 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.722128 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.709020 12757 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.727813 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:30.702651 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.717691 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.722718 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.678897 11471 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:30.725088 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:30.731658 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97006.8us
I20260709 13:15:30.732213 11543 raft_consensus.cc:2410] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:15:30.732641 11544 raft_consensus.cc:2368] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4.
I20260709 13:15:30.732968 11475 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.731284 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.722954 11542 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.718245 12819 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.718047 11298 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.710453 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:30.726892 12757 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.725219 11476 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.747324 12785 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.748335 12874 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.688949 12894 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:15:30.750454 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 100) took 1302 ms. Trace:
I20260709 13:15:30.803804 11396 rpcz_store.cc:276] 0709 13:15:29.448078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.505214 (+1057136us) service_pool.cc:224] Handling call
0709 13:15:30.505538 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.709336 (+203798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.750185 (+ 40849us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54003d4ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:30.750364 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358805376}
I20260709 13:15:30.743521 12732 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.736232 12893 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.752305 12116 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:30.752959 12919 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:15:30.743536 12863 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.756513 12831 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.757934 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:30.830464 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 98202.2us
I20260709 13:15:30.831724 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.833353 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:30.834367 12883 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.834712 12883 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:30.835031 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.835855 11550 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:30.765352 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 185980us
I20260709 13:15:30.816015 12732 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.771318 12687 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:30.774274 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 128464us
I20260709 13:15:30.777712 11547 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.778131 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.729646 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:30.843413 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 74259.7us
I20260709 13:15:30.844508 11475 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:30.845678 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.784188 11477 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:30.743037 12804 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:30.780736 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:30.792843 12764 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.859707 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.796973 12806 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.797883 12903 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.797739 12884 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:30.798750 12928 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.799665 12712 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.874475 11551 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.801816 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:30.843551 12785 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.777769 11549 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.825937 12863 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.767938 12839 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:30.885959 12839 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:30.886389 12839 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:30.816232 12893 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:30.895769 12893 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.893778 12804 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.900379 12712 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:30.902073 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:30.903255 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.904556 12874 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.905179 12874 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:30.905627 12874 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:30.906212 12766 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:30.906518 12874 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.906827 12766 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.908869 12766 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:30.861763 12915 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:15:30.910122 12757 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.841602 12732 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.839473 12883 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.875252 11473 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.800324 11298 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.861956 12862 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:30.848115 12698 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.884429 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.862097 12112 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.925946 12839 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.927166 12831 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:30.912782 12786 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:30.906405 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:30.934823 12119 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.872104 11476 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:30.920121 11298 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
W20260709 13:15:30.922528 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 242918us
W20260709 13:15:30.916733 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46146.2us
W20260709 13:15:30.940510 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.893010 12884 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.878268 12785 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.940660 12118 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.946897 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:30.949330 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 295069us
I20260709 13:15:30.945300 11298 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.953028 12115 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.952842 11298 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:15:30.954329 11298 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:30.949983 12785 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.958171 12903 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:30.959105 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 125844us
I20260709 13:15:30.960636 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.962334 12712 raft_consensus.cc:2764] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:30.934430 12786 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:30.963933 12786 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:30.965776 12786 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.968425 12114 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:15:30.969951 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.971485 12804 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:30.910158 12915 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:30.972472 12915 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:30.965813 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:30.973310 12915 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:30.974799 12915 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.976094 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.976755 11545 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.979017 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.979364 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.979717 11549 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.980132 11542 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.980662 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.980996 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.981346 11549 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:30.982123 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.982206 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.983343 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.995077 11549 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:30.995716 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.996533 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:30.997272 11547 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:30.998425 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:30.999495 11544 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.001410 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:31.005918 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55304.8us
I20260709 13:15:31.012185 11551 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.015127 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.015753 11547 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.015599 12732 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.019997 12883 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.022140 12883 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.022740 12732 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:31.027129 12687 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.030742 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 83968us
I20260709 13:15:31.018747 11473 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:31.040442 12119 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:31.031716 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:30.983767 12839 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.037562 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:31.045418 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44115.6us
I20260709 13:15:31.048542 11547 raft_consensus.cc:2410] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:15:31.051854 12804 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.051956 12883 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.029410 12862 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.055182 11549 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.052436 12883 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.969076 12934 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.057013 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.060702 11396 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.064940 12923 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.065229 12944 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.065505 12923 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:31.066622 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 237665us
I20260709 13:15:31.066501 12923 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.069100 12924 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.039111 12903 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:31.072185 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.038380 12915 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.073815 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.060883 12883 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.062876 11542 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.081485 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:31.086637 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.087442 12832 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:15:31.087761 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.088533 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.089742 11365 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.091150 12884 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.091234 11394 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.091954 12884 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:31.092327 12884 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:31.092698 12884 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.095384 12732 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.090927 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.039356 12764 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:15:31.101177 11396 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.101006 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:31.100790 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:31.107249 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.107846 12806 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:31.108337 11402 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:15:31.110105 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.110736 11401 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.119347 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.114069 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:31.116086 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.052472 12804 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.077919 12924 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:31.142931 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 52050.5us
I20260709 13:15:31.084955 12915 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.144611 11545 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:15:31.135962 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.088562 11400 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.147769 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.153905 12766 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.154433 12766 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.156220 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.156908 12766 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:31.156761 12819 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.158088 11396 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.158157 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.158942 11473 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
W20260709 13:15:31.120651 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.110924 12940 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:15:31.060072 12934 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.161158 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.168987 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.063877 12933 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 13:15:31.112519 12832 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.177754 12119 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:31.115425 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.115088 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.146872 12874 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.082858 12903 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.115296 12732 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.176146 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.115594 12757 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.180441 11401 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:31.151522 12712 raft_consensus.cc:2764] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:31.196451 12712 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.160260 12819 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.190414 11550 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.172798 12766 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.159278 12915 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.203487 11550 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.111155 12937 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:15:31.205924 12933 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:31.206319 12937 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:31.206848 12937 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:31.207929 12112 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:31.207901 12937 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.209364 12937 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.211668 12804 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.212257 12941 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:15:31.213127 12716 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:31.213531 12716 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:31.213903 12716 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.215127 12915 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.206389 12933 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.112186 12785 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.177990 12832 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.224138 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:31.236254 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46074us
I20260709 13:15:31.237504 11395 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:31.245368 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:31.252418 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 101445us
I20260709 13:15:31.245478 12117 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:31.248492 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.221768 12874 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.236902 12762 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:31.235841 12944 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.260217 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:31.262181 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 104108us
I20260709 13:15:31.261831 12762 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.266834 12762 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.267499 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.268155 12762 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.268296 12757 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.268929 12757 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:31.269835 12757 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.271050 11366 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.272020 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.275820 12950 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.276522 12950 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.276970 12950 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:31.277848 12950 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.271149 12863 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.278612 12732 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.284102 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.284863 11544 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:31.288254 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.289716 12832 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.291278 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:31.292047 11542 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.292973 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.289705 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.318420 11475 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.303736 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.305979 12903 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:31.308319 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:31.309430 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43806.8us
I20260709 13:15:31.264421 12893 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.295595 12874 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.243343 12924 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.316382 12915 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.318481 11544 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:31.321905 11549 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.348860 11473 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:15:31.317919 12945 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:15:31.356918 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.357096 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.369161 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.369913 11550 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.371475 12942 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
W20260709 13:15:31.379235 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.389325 12927 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:31.391431 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.388011 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:31.395509 12927 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:31.395751 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.403374 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.274866 12884 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.404407 11549 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:31.361078 12894 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.414767 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.415551 11477 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:31.418917 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72740.8us
I20260709 13:15:31.419759 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.420209 11476 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:31.410389 11550 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:31.411140 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.412884 11475 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.423857 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.427057 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.427655 11473 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.413957 12939 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.258s	user 0.003s	sys 0.000s
I20260709 13:15:31.439488 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.362998 12762 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:15:31.405699 12837 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.396526 12927 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.444465 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.414327 12832 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.413985 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.403774 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.449661 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.422600 12712 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.459834 11476 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.457325 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.459308 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.460852 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.461645 11401 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.437201 12698 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.463443 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.437194 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.464665 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.343323 12933 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.471524 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.472549 11367 leader_election.cc:400] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4.
I20260709 13:15:31.477072 11473 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.480178 11367 leader_election.cc:403] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.484179 12762 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.485937 12883 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.487118 12115 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.487979 12884 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:31.488194 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.488621 12687 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:31.489044 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.489018 12687 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.489624 11394 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:31.489907 12687 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.490386 12951 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.491182 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:31.492023 11393 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:31.492187 12933 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.492789 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.493602 12114 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:31.495059 12804 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:31.495303 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.495985 11395 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:31.491267 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.426533 12894 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.496814 11401 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.456478 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.498956 11477 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.501245 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.504786 12118 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.463340 12837 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.515301 11473 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:31.470049 12941 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.473318 12927 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.517347 11476 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.517225 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.528527 11477 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.478147 11472 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.478781 12785 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.534787 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.491269 11400 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:31.469226 12934 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:31.497311 12883 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.538269 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:31.530661 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.496536 12804 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:15:31.540983 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:31.541545 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:31.540726 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.560899 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:31.561182 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.589679 11396 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:31.545265 11395 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.516727 12884 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.563943 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:31.564441 12936 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:15:31.562336 12113 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:31.579283 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.529459 12716 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.620535 12716 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:31.621306 11394 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:31.565927 12712 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:31.622223 12712 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.624588 11472 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:31.569206 12932 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:31.625429 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 85936.7us
I20260709 13:15:31.625558 12932 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.626765 12883 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.626520 12932 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.626185 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.627970 11477 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.571372 12698 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:31.575273 12893 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:31.576853 12949 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:15:31.631424 12698 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.578452 12863 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.579247 11402 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:31.562522 11399 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:31.583797 11475 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:31.584328 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.633908 11476 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.634884 12883 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.567641 12837 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:31.639287 12716 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:31.603231 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 161944us
I20260709 13:15:31.640875 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.612792 12894 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:31.647513 11401 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:31.664245 11476 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.665870 11395 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:31.669135 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.670948 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:31.673013 12123 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:31.674029 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.674415 11542 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:31.674618 11475 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.675890 12893 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.676910 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.677467 11469 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.678448 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:31.679142 11393 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:31.679991 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.681082 12956 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.681555 12956 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:31.687659 11365 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.690074 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:31.693661 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 190561us
I20260709 13:15:31.694440 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:31.696962 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.700675 11475 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:31.682384 12956 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.707463 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 116344us
I20260709 13:15:31.703164 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.702280 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.704777 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.708154 11477 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.711328 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.712486 11367 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:31.690125 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.716048 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.721621 11475 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.717557 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.704160 12944 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.740695 11469 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.728648 11477 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.730414 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.720826 12806 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.716593 12915 raft_consensus.cc:2764] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:31.746474 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:31.752547 11476 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.752775 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.748785 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 282161us
I20260709 13:15:31.756315 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:31.757696 11469 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.759532 12757 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.737923 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:31.760601 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 58378.8us
I20260709 13:15:31.743340 12893 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.742892 12785 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.770792 11475 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.758384 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:31.719408 12698 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.792207 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 52027.1us
I20260709 13:15:31.793006 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:31.749116 12716 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:31.796868 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66545.4us
I20260709 13:15:31.800284 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.800604 11396 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.760675 12806 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:31.762039 11470 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:31.808687 12944 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.826421 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.836418 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:31.836560 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:31.837342 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.837409 12953 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.800053 12757 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:31.834514 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:31.837973 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 337617us
W20260709 13:15:31.842748 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.828258 11472 raft_consensus.cc:2368] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3.
I20260709 13:15:31.808911 12893 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:31.860494 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.861989 12085 leader_election.cc:400] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:31.809695 12970 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.862344 12085 leader_election.cc:403] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.871789 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.828133 12115 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.868381 11476 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:31.877746 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:31.882622 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46170.7us
I20260709 13:15:31.872907 11400 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:15:31.874699 12959 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:15:31.896624 12855 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:31.897076 12855 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.878685 12924 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.840782 12915 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:15:31.883271 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.870035 12716 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.895123 12785 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:31.906454 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.907025 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.907248 12785 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.908488 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.909233 12915 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.909694 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.910511 11547 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:31.910621 11546 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:31.912596 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:31.918123 12855 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:31.953745 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 126284us
I20260709 13:15:31.911281 11400 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.885660 12970 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:31.965088 11397 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:31.965142 12972 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:15:31.965972 12970 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:31.967916 12974 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:31.969996 12716 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:31.971668 12980 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.972759 12980 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:31.973317 12980 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:31.979156 12806 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:31.980274 12952 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.552s	user 0.003s	sys 0.001s
I20260709 13:15:31.901747 12975 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.957129 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:31.897260 12929 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:31.958446 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 80767.5us
I20260709 13:15:31.984330 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:31.961644 12973 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:15:31.962558 12965 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:15:31.963567 12971 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:15:31.992403 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 109173us
I20260709 13:15:31.993429 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.994616 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1287) took 1017 ms. Trace:
I20260709 13:15:31.994324 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:31.956616 12976 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.995154 11543 rpcz_store.cc:276] 0709 13:15:30.977309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.035744 (+ 58435us) service_pool.cc:224] Handling call
0709 13:15:31.054128 (+ 18384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.079959 (+ 25831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.079991 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:31.127248 (+ 47257us) log.cc:844] Serialized 26 byte log entry
0709 13:15:31.250014 (+122766us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:31.250069 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.250160 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.979475 (+729315us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:31.983776 (+  4301us) raft_consensus.cc:1685] finished
0709 13:15:31.983804 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.994533 (+ 10729us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10001,"raft.queue_time_us":10412,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":11813760,"thread_start_us":64711,"threads_started":1,"wal-append.queue_time_us":189583}
I20260709 13:15:31.903697 12893 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.010048 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 131137us
I20260709 13:15:32.011691 12085 leader_election.cc:400] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:32.012343 11397 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:32.013489 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:31.974435 12980 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:31.909823 12915 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.021509 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:32.032958 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.034720 12806 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:32.035238 12806 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:32.035502 12855 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.036242 12806 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.037467 12806 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.037671 12855 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.044095 12855 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.044621 12855 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:32.045604 12855 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.038437 12984 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.047317 12984 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.048333 12984 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.052356 12915 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.073429 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 52039.4us
I20260709 13:15:32.027207 12698 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.032382 12785 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.031450 12953 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:32.078670 11514 leader_election.cc:336] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.751s (SENT)
I20260709 13:15:32.090294 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:32.081143 12884 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4.
I20260709 13:15:32.030531 12982 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:32.027434 12944 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.069821 12929 raft_consensus.cc:2764] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:15:32.073138 12965 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:32.002693 12757 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:32.102618 12960 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.103363 12766 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:32.103240 12960 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.105376 12960 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:32.107183 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.108054 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.118530 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.119546 12960 raft_consensus.cc:2764] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:32.099273 12785 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.122809 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:32.142241 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.131196 12785 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.141543 12953 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.106637 12976 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:32.148571 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 74352.3us
I20260709 13:15:32.149112 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.151856 12960 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.152408 12960 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:32.153322 12960 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.155256 12944 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.157578 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 416840us
I20260709 13:15:32.158608 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:32.148622 12712 consensus_meta.cc:287] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.003s
I20260709 13:15:32.168105 12712 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.168597 12929 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:32.169771 11442 leader_election.cc:400] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4.
I20260709 13:15:32.170148 11442 leader_election.cc:403] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:32.171402 12114 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:15:32.170620 12712 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:32.173604 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.167862 12982 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:32.175824 12927 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
W20260709 13:15:32.181892 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 187486us
I20260709 13:15:32.188230 12119 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:32.188414 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.174769 12929 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4.
I20260709 13:15:32.193291 12976 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.201833 11393 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:32.201424 12982 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.202087 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:32.202626 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.216689 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.222389 12883 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.231671 11365 leader_election.cc:400] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3.
I20260709 13:15:32.232076 11365 leader_election.cc:403] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:32.232139 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:32.202718 12993 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.237532 11395 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:32.208459 12982 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.220027 12884 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.223090 12883 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.240037 12993 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.247684 12884 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.207819 12991 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:15:32.239610 12928 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:32.247363 12832 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.249246 12883 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.253512 12832 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.253258 12884 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.238025 12951 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3.
I20260709 13:15:32.258163 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:32.262002 12687 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.275475 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:32.276304 11473 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.292713 12993 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.292986 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:32.293532 11468 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.260725 12975 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3.
I20260709 13:15:32.256424 12712 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.295428 12712 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.261530 12929 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.312991 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:32.314414 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:32.313978 12937 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.320441 12832 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:32.323246 11468 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:32.325933 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.328207 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.329006 11393 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:32.321168 12982 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.311874 12929 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:32.319227 12928 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.334317 12928 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:32.334826 12928 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.317133 12937 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:32.251775 12894 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:32.325228 12832 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.351488 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.327510 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:32.332544 12929 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.332293 12980 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.372977 12894 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.381047 11473 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:32.383795 12903 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.345634 12937 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.388629 12712 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:32.403023 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.408645 12855 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.409235 12855 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.411394 12855 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.368634 12832 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.424698 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.395509 12903 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.425609 11393 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:32.425884 12832 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:32.389725 11472 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:32.438140 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41380 (request call id 93) took 1151 ms. Trace:
I20260709 13:15:32.438428 11550 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:32.438758 11542 rpcz_store.cc:276] 0709 13:15:31.286934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.292663 (+  5729us) service_pool.cc:224] Handling call
0709 13:15:32.437860 (+1145197us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400134cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:32.438048 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709923200}
I20260709 13:15:32.448691 12687 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.449443 12687 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.453236 12955 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.877s	user 0.005s	sys 0.002s
W20260709 13:15:32.454461 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.463315 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.464211 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:32.467267 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.469368 12933 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.469949 12967 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.622s	user 0.002s	sys 0.000s
I20260709 13:15:32.470057 12933 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.472625 12766 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.473326 12766 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.475796 11551 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:32.480018 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:32.481206 11543 raft_consensus.cc:2410] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:32.494773 12855 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:32.538373 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:32.531615 11549 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:32.396148 12980 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:32.532027 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:32.561115 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:32.582039 11548 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.573918 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:32.585170 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 357) took 1157 ms. Trace:
I20260709 13:15:32.585609 12119 rpcz_store.cc:276] 0709 13:15:31.419234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.420800 (+  1566us) service_pool.cc:224] Handling call
0709 13:15:32.576792 (+1155992us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b540047ca54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:32.577003 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719354240}
I20260709 13:15:32.589243 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.495841 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:32.543427 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.552376 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 88278.3us
I20260709 13:15:32.597565 11551 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:32.607928 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:32.610874 11469 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:32.612231 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:32.621328 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.623394 12119 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:32.625695 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:32.530571 12894 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:32.629853 12956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.028s	sys 0.055s
I20260709 13:15:32.638051 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:32.631919 11549 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:32.639673 11549 pending_rounds.cc:85] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 1
I20260709 13:15:32.638913 11476 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:32.640588 12123 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:32.528225 12687 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:32.495144 12933 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:32.648288 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.626530 11546 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:32.595779 12698 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.656483 11551 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:32.542340 12832 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:32.657435 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.658259 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.532203 12893 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:15:32.667774 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:32.565663 12732 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:15:32.621719 12855 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:32.654127 12956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.028s	sys 0.055s
I20260709 13:15:32.659332 11549 pending_rounds.cc:107] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:32.697263 11549 raft_consensus.cc:2889] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.697427 12996 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.698084 12766 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:32.632134 12950 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.698650 12950 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.699132 12950 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.687549 12839 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.667716 12687 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.689924 12995 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.712280 12995 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.693221 12968 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.715467 12968 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.618404 12991 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:15:32.658600 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:32.701629 12839 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.685891 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 98) took 1309 ms. Trace:
I20260709 13:15:32.684640 12832 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.707931 12732 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:32.696028 12980 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.718192 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.723114 11548 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:32.710476 12687 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.725317 12929 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.735175 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.012s	sys 0.039s
I20260709 13:15:32.685362 12933 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:15:32.681448 12698 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:32.726457 12766 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:32.739617 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.012s	sys 0.039s
I20260709 13:15:32.690874 12894 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:15:32.751747 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 738340us
I20260709 13:15:32.733496 12115 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:32.730569 11474 rpcz_store.cc:276] 0709 13:15:31.359040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.361342 (+  2302us) service_pool.cc:224] Handling call
0709 13:15:31.445629 (+ 84287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.482595 (+ 36966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.482632 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:31.483206 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:15:31.713525 (+230319us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:31.716362 (+  2837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.728891 (+ 12529us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:32.469087 (+740196us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:32.605803 (+136716us) raft_consensus.cc:1685] finished
0709 13:15:32.605822 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.658323 (+ 52501us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5,"raft.queue_time_us":101507,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":17289344,"thread_start_us":212390,"threads_started":1,"wal-append.queue_time_us":312334}
I20260709 13:15:32.756449 11401 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:32.733234 12114 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:32.766606 11468 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:32.736451 12984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.010s	sys 0.042s
I20260709 13:15:32.756084 12766 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:32.771289 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:32.772130 12119 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:32.755317 12698 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.730443 12996 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:32.757817 12764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.015s	sys 0.023s
W20260709 13:15:32.769256 12970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.026s	sys 0.033s
I20260709 13:15:32.812232 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:32.794536 12984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.010s	sys 0.042s
I20260709 13:15:32.768563 12766 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.757118 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:32.857750 12766 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.859185 12923 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.859477 12903 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:32.860067 12903 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.860500 12903 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.863204 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 106211us
W20260709 13:15:32.816867 12970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.027s	sys 0.033s
W20260709 13:15:32.806540 12987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.010s	sys 0.028s
I20260709 13:15:32.810767 12832 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:32.857779 12839 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:32.876650 12928 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 13:15:32.878386 12929 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.880203 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:32.881319 13001 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:32.888639 12996 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:32.818694 12764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.016s	sys 0.023s
W20260709 13:15:32.800496 12785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.016s	sys 0.024s
W20260709 13:15:32.952996 12987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.010s	sys 0.035s
W20260709 13:15:32.951472 12932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.275s	user 0.015s	sys 0.042s
W20260709 13:15:32.928541 12915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.029s	sys 0.043s
I20260709 13:15:32.810922 12855 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:32.949800 12951 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.037127 12932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.016s	sys 0.043s
I20260709 13:15:32.975649 12839 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:33.045724 12903 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:33.047472 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.048550 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 184445us
W20260709 13:15:33.027544 12785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.016s	sys 0.024s
I20260709 13:15:32.993633 11542 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:33.049275 12915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.029s	sys 0.044s
I20260709 13:15:33.083560 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:33.084344 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.995726 12968 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:32.951179 12928 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.108104 12951 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.112824 12995 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.057813 12839 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.132894 12839 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.099393 12950 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.099825 12923 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:15:33.141312 12937 consensus_meta.cc:287] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.738s	user 0.005s	sys 0.004s
I20260709 13:15:33.141722 12937 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.142794 11475 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:33.144451 12928 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:33.126220 12117 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:33.149477 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:33.153851 11542 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:33.148197 12951 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.158950 12928 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:33.159520 12928 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.159880 12928 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:33.167364 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 95404.5us
W20260709 13:15:33.171686 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.138832 12950 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.158674 12995 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.246064 12937 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:33.167092 12687 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:33.253132 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:33.263978 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:33.306668 12950 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:33.324100 12115 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:33.324568 12950 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:33.263403 11470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.028s	sys 0.055s
I20260709 13:15:33.328979 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:33.334142 12804 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:33.335050 12927 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:33.334831 12804 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.335620 12927 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.253203 12995 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:33.330082 11470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.028s	sys 0.055s
W20260709 13:15:33.326364 12944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.164s	user 0.035s	sys 0.036s
W20260709 13:15:33.352664 12944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.194s	user 0.036s	sys 0.037s
W20260709 13:15:33.631898 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.454465 11471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.012s	sys 0.071s
W20260709 13:15:33.637065 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.341s	user 0.018s	sys 0.084s
W20260709 13:15:33.559288 12982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.217s	user 0.002s	sys 0.041s
W20260709 13:15:33.559540 11543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.010s	sys 0.012s
W20260709 13:15:33.559818 11366 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.012s	sys 0.012s
W20260709 13:15:33.496811 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.182s	user 0.021s	sys 0.058s
W20260709 13:15:33.648665 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.333s	user 0.027s	sys 0.068s
W20260709 13:15:33.650367 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 1.33592e+06us
W20260709 13:15:33.559710 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.012s	sys 0.010s
W20260709 13:15:33.637434 12893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.000s	sys 0.012s
W20260709 13:15:33.651541 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.015s	sys 0.012s
W20260709 13:15:33.444319 12894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.000s	sys 0.002s
W20260709 13:15:33.353554 12883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.043s	sys 0.052s
W20260709 13:15:33.416991 12837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.208s	user 0.018s	sys 0.062s
I20260709 13:15:33.436097 12114 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:33.643640 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.000s	sys 0.027s
W20260709 13:15:33.644784 12874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.256s	user 0.017s	sys 0.024s
I20260709 13:15:33.668962 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:33.660378 12894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.002s	sys 0.027s
W20260709 13:15:33.531518 12960 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.216s	user 0.015s	sys 0.029s
W20260709 13:15:33.656523 11543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.010s	sys 0.016s
W20260709 13:15:33.497699 12976 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.005s	sys 0.044s
W20260709 13:15:33.508230 12884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.004s	sys 0.037s
I20260709 13:15:33.663017 12950 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:33.656203 11366 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.012s	sys 0.017s
W20260709 13:15:33.685070 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 1.20646e+06us
I20260709 13:15:33.687433 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:33.665647 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.000s	sys 0.027s
W20260709 13:15:33.497644 12993 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.177s	user 0.004s	sys 0.035s
I20260709 13:15:33.629858 12984 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:33.612382 12757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.017s	sys 0.013s
W20260709 13:15:33.447544 12953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.298s	user 0.027s	sys 0.099s
I20260709 13:15:33.407228 12997 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.654197 12893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.001s	sys 0.012s
W20260709 13:15:33.658974 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.684165 12884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.007s	sys 0.037s
W20260709 13:15:33.659832 12982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.004s	sys 0.044s
I20260709 13:15:33.679713 12937 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:33.661597 12883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.393s	user 0.055s	sys 0.065s
I20260709 13:15:33.648809 12804 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:33.662915 12837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.454s	user 0.030s	sys 0.072s
I20260709 13:15:33.534152 12928 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:33.671607 12874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.017s	sys 0.024s
W20260709 13:15:33.710721 11517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.111s	user 0.006s	sys 0.032s
W20260709 13:15:33.715821 11517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.008s	sys 0.035s
W20260709 13:15:33.670197 11472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.006s	sys 0.017s
W20260709 13:15:33.688400 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 1.07906e+06us
I20260709 13:15:33.720201 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:33.695782 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.017s	sys 0.027s
W20260709 13:15:33.720760 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.017s	sys 0.028s
I20260709 13:15:33.642921 12927 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:33.686816 11366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 2.077s (SENT) [suppressed 39 similar messages]
W20260709 13:15:33.683279 12976 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.011s	sys 0.044s
W20260709 13:15:33.672318 11514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.014s	sys 0.011s
W20260709 13:15:33.692443 12993 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.008s	sys 0.037s
W20260709 13:15:33.741195 13009 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:15:33.744750 13012 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:33.746189 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1335) took 1472 ms. Trace:
W20260709 13:15:33.692307 12757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.017s	sys 0.013s
W20260709 13:15:33.692073 12953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.542s	user 0.028s	sys 0.101s
I20260709 13:15:33.746711 11470 rpcz_store.cc:276] 0709 13:15:32.273794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.290811 (+ 17017us) service_pool.cc:224] Handling call
0709 13:15:32.291217 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.291712 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.291745 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.292253 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.378684 (+1086431us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:33.385002 (+  6318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.385103 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.745826 (+360723us) raft_consensus.cc:1685] finished
0709 13:15:33.745855 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.746105 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2383,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":1059085,"threads_started":1,"wal-append.queue_time_us":1360790}
I20260709 13:15:33.684336 12894 consensus_queue.cc:1243] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:33.694096 12995 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:33.704016 12112 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.030s	sys 0.012s
W20260709 13:15:33.681566 12960 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.378s	user 0.015s	sys 0.032s
W20260709 13:15:33.726001 11368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.007s	sys 0.014s
W20260709 13:15:33.731434 11514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.014s	sys 0.011s
W20260709 13:15:33.732257 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 45479.6us
I20260709 13:15:33.739472 12687 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:15:33.609216 12955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.154s	user 0.015s	sys 0.022s
I20260709 13:15:33.720855 12937 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:33.762948 11440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.007s	sys 0.020s
W20260709 13:15:33.768610 11440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.008s	sys 0.020s
W20260709 13:15:33.769299 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.753756 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.719350 11472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.007s	sys 0.017s
I20260709 13:15:33.755849 11545 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 4. Ops: []
W20260709 13:15:33.755897 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67278.6us
W20260709 13:15:33.761348 12112 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.030s	sys 0.017s
W20260709 13:15:33.755285 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.765522 11368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.007s	sys 0.014s
I20260709 13:15:33.773834 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:33.775214 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:33.775997 11549 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:33.762709 12894 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:33.767241 12955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.312s	user 0.016s	sys 0.023s
I20260709 13:15:33.711048 12984 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.708690 12997 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.778582 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:33.781767 11549 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:33.788131 11551 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:33.788355 11549 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:33.793239 11544 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:33.762319 12837 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:15:33.782037 12894 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.793422 11549 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.781014 12960 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.785878 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:33.800004 11443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.210s	user 0.007s	sys 0.012s
I20260709 13:15:33.779603 12923 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 13:15:33.832100 11443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.007s	sys 0.013s
W20260709 13:15:33.771303 12933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.010s	sys 0.017s
I20260709 13:15:33.783434 12804 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:15:33.800148 12764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.002s	sys 0.016s
W20260709 13:15:33.802862 12980 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.009s	sys 0.010s
I20260709 13:15:33.844282 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:33.850014 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:33.850581 11551 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:33.800134 12996 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.009s	sys 0.011s
W20260709 13:15:33.825337 12961 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.005s	sys 0.021s
I20260709 13:15:33.851684 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:33.841907 12855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.011s	sys 0.025s
I20260709 13:15:33.801622 12894 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.800269 13038 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.855311 11475 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:33.844897 12933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.011s	sys 0.017s
I20260709 13:15:33.918812 12933 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.919404 12933 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:33.851028 12764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.016s
I20260709 13:15:33.920926 12764 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:33.850405 12980 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.009s	sys 0.010s
W20260709 13:15:33.921366 13005 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:15:33.800352 12944 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.922004 12944 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.924464 12944 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.798187 12893 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.925109 12893 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:33.927315 11474 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:33.927574 13018 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:15:33.927706 11474 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:33.928647 11474 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:33.929402 13011 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:15:33.929637 11474 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:33.930923 12999 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.672s	user 0.007s	sys 0.000s
I20260709 13:15:33.931752 11476 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:33.914314 12997 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.932852 12997 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:33.843669 12687 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:15:33.798556 13026 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:33.878973 12976 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.935861 12976 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:33.937230 12687 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:33.938717 12976 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:33.914498 12928 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:33.939539 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:33.881016 12855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.026s
I20260709 13:15:33.912257 13031 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:33.874661 12961 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.005s	sys 0.022s
I20260709 13:15:33.913414 12960 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.913671 13034 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:33.913671 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:33.844122 13030 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:33.856045 12996 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.009s	sys 0.012s
I20260709 13:15:33.918612 12883 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.830655 13033 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:15:33.963032 11473 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:33.967324 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:33.941146 12115 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:33.999563 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.016415 11475 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:33.939495 13026 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.010948 11476 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.921391 12764 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:33.927172 12893 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:33.967854 13038 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.010701 12883 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.020022 13030 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.021080 11551 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.021205 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:34.021607 13014 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:15:34.022663 13006 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:15:34.024906 11549 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:34.029671 12839 consensus_meta.cc:287] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.001s
I20260709 13:15:33.970871 13034 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:34.031947 13037 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.032495 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 104) took 1577 ms. Trace:
I20260709 13:15:33.845888 12937 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.048041 11551 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:34.048377 11473 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:34.069082 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1304) took 1570 ms. Trace:
I20260709 13:15:34.044322 12855 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.067535 12960 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.108071 11551 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.099453 13037 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:34.118424 11551 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.123896 11476 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:34.125968 11549 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:34.136389 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.137193 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 365267us
I20260709 13:15:34.110350 13016 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.138273 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:34.148778 11476 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:34.113656 12927 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.099975 12839 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.168951 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1338) took 1894 ms. Trace:
I20260709 13:15:34.169453 11471 rpcz_store.cc:276] 0709 13:15:32.274826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.295069 (+ 20243us) service_pool.cc:224] Handling call
0709 13:15:32.295415 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.295786 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.295812 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.296263 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.637611 (+1341348us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:33.637655 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.637728 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.168662 (+530934us) raft_consensus.cc:1685] finished
0709 13:15:34.168690 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.168870 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3967,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":9935,"thread_start_us":1340972,"threads_started":1,"wal-append.queue_time_us":1501726}
I20260709 13:15:34.173576 11473 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:34.174448 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.171787 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:34.182119 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.183387 11474 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:34.183831 11474 pending_rounds.cc:85] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 1
I20260709 13:15:34.184067 11474 pending_rounds.cc:107] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:34.184317 11474 raft_consensus.cc:2889] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:34.149488 12928 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.170686 13015 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:15:34.202005 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 332) took 1565 ms. Trace:
I20260709 13:15:34.183522 12989 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.204223 11468 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:34.204838 12687 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.206667 11551 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:34.207268 13044 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:34.116987 12960 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.207363 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 86) took 2956 ms. Trace:
I20260709 13:15:34.150522 12893 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.117762 12855 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.130045 12764 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.151872 13034 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:34.230782 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 382754us
I20260709 13:15:34.395833 11551 pending_rounds.cc:85] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 1
W20260709 13:15:34.262583 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.259636 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 87962.6us
I20260709 13:15:34.397009 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.396590 11551 pending_rounds.cc:107] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:34.398524 11402 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:34.150223 12996 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.372400 13043 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:15:34.411840 11395 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:15:34.255317 13042 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:15:34.175314 13054 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.295646 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:34.418929 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 123364us
W20260709 13:15:34.401276 11514 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.000s	sys 0.027s
W20260709 13:15:34.393674 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.425820 11514 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.000s	sys 0.027s
I20260709 13:15:34.402460 13063 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:34.398449 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.395435 12989 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:34.403863 11366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.001s	sys 0.028s
I20260709 13:15:34.455651 11399 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.168499 12984 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.295838 12874 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:34.414525 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.003s	sys 0.053s
W20260709 13:15:34.395771 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 617234us
I20260709 13:15:34.400604 12687 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.417237 11398 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:34.393242 12832 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.405181 12924 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:34.398468 11551 raft_consensus.cc:2889] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:34.426159 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 648632us
I20260709 13:15:34.293680 11470 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:34.255379 13029 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.395895 11545 rpcz_store.cc:276] 0709 13:15:31.251234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.251347 (+   113us) service_pool.cc:224] Handling call
0709 13:15:31.251622 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.770517 (+2518895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.207108 (+436591us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b540041dcd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:34.207282 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5758716672}
I20260709 13:15:34.461917 13055 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:34.460557 11366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.003s	sys 0.029s
I20260709 13:15:34.435832 11547 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:34.485595 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.486907 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.459381 13034 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.167361 13016 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:34.242482 13028 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:34.462514 11475 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:34.191704 12837 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.428067 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:34.469255 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:34.470016 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.003s	sys 0.053s
I20260709 13:15:34.465605 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.283439 13051 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.180s
I20260709 13:15:34.414920 12764 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.255577 12933 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:34.477818 11546 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:15:34.406188 12928 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.410399 12996 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.482501 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.275609 13050 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:34.462625 12993 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.427517 11402 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:34.560532 12993 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.489019 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.561529 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.466378 13030 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.565673 13030 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.461834 13026 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.465631 12884 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.566210 12993 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.521770 13028 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.101591 11472 rpcz_store.cc:276] 0709 13:15:32.455430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.466492 (+ 11062us) service_pool.cc:224] Handling call
0709 13:15:32.639322 (+172830us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.702387 (+ 63065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.715831 (+ 13444us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.716580 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.785251 (+1068671us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:33.785818 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.795553 (+  9735us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.032259 (+236706us) raft_consensus.cc:1685] finished
0709 13:15:34.032293 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.032433 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103583,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":7419,"thread_start_us":1055523,"threads_started":1,"wal-append.queue_time_us":1245158}
I20260709 13:15:34.566757 12884 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.102725 11543 rpcz_store.cc:276] 0709 13:15:32.488226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.606332 (+118106us) service_pool.cc:224] Handling call
0709 13:15:32.606725 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.607023 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:33.682065 (+1075042us) log.cc:844] Serialized 14 byte log entry
0709 13:15:34.057758 (+375693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:34.057830 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:34.057889 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.057976 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.058138 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11249,"prepare.queue_time_us":1157827,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2182,"raft.queue_time_us":9272,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":33068,"thread_start_us":1415163,"threads_started":2}
I20260709 13:15:34.569139 12884 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.569293 11543 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:34.469852 12874 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.570050 12874 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.503929 13068 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.506634 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.396410 12112 rpcz_store.cc:276] 0709 13:15:32.630328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.658394 (+ 28066us) service_pool.cc:224] Handling call
0709 13:15:32.730701 (+ 72307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.748418 (+ 17717us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:33.789374 (+1040956us) log.cc:844] Serialized 14 byte log entry
0709 13:15:34.130818 (+341444us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:34.145078 (+ 14260us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:34.147976 (+  2898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.169868 (+ 21892us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.195190 (+ 25322us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":634483,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":148,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":21330,"thread_start_us":1279761,"threads_started":2,"wal-append.queue_time_us":512966}
I20260709 13:15:34.507434 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.507328 12989 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.508303 12883 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:34.508442 13062 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:15:34.475594 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.508647 13016 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.481102 12984 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:15:34.475917 12687 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.409000 12893 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.642190 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.497563 11401 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.505760 12960 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:34.523144 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.475265 12832 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:34.524883 12837 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:34.497831 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.476035 13045 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:34.484328 13055 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.407078 13044 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.540261 11546 pending_rounds.cc:85] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 2
I20260709 13:15:34.658052 11546 pending_rounds.cc:107] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:15:34.658399 11546 raft_consensus.cc:2889] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:34.490324 13029 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.490644 12839 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.543752 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.543817 11545 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:34.555688 13060 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:34.460182 13054 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.560460 13050 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.463220 12757 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:34.484651 12924 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.562208 11402 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:34.303754 13037 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.680078 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.646564 11393 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:34.637557 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.635665 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 146072us
I20260709 13:15:34.566268 13064 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:34.418123 13038 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:34.641243 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 171503us
I20260709 13:15:34.644883 11366 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:34.650308 11475 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:34.651556 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:34.654608 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.668002 11471 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:34.566253 13030 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.634199 12961 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:34.641054 12883 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.670307 11398 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:34.637039 13031 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:34.647348 12993 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.783010 12928 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.783507 13052 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:34.785246 12893 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.659165 13055 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:34.792076 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.535167 12970 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:15:34.642098 12884 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.794852 13028 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:34.812651 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.842406 11402 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:34.844318 11549 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:34.572541 13068 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.851243 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.886827 11475 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:34.851910 11477 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:34.852681 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.863408 13056 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:15:34.865094 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 220116us
I20260709 13:15:34.670362 11468 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:34.680817 12960 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.669550 12989 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.669307 12855 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.791566 13026 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:34.812179 13057 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.809188 11469 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
W20260709 13:15:34.813023 12894 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.001s
I20260709 13:15:34.885370 12112 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:34.900939 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:34.902249 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.904646 13044 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:34.905321 12982 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.936147 12982 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.936586 12982 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.937407 13030 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:34.940176 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.907522 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:34.909794 11475 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:34.878794 11472 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:34.915504 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:34.944940 11550 raft_consensus.cc:2410] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:34.920181 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.947029 13057 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.871060 13031 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.952527 12996 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.659821 13045 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.902902 13033 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.923283 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 278461us
I20260709 13:15:34.925755 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.933357 13083 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:15:34.991278 13078 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.991436 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:34.992120 11546 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:34.991928 13078 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.934712 13044 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:34.993384 13044 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.993886 13044 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.996003 13044 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.996208 12995 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.935389 13029 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.935644 13056 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.997257 13056 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:34.998241 13056 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.998793 13076 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.999627 13056 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.999363 13076 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:34.954560 12982 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:35.014539 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 175257us
I20260709 13:15:34.964377 13054 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.966626 13050 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.972841 13068 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:34.853669 13064 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.973819 11470 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:34.877511 12993 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.914788 13026 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.914973 12855 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:15:34.981837 12924 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.983171 11469 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:34.870549 12961 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.870397 12883 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:34.891834 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.996625 12995 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:34.990669 11399 consensus_meta.cc:287] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.002s
I20260709 13:15:34.906144 12960 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.873757 13052 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.009543 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.010121 13044 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.076712 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.012580 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:34.893240 12884 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.874907 12832 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.016002 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:35.085253 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:35.085551 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 69637.7us
I20260709 13:15:35.086126 12123 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:35.014237 13033 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.001718 13076 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:34.950850 13057 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:34.810738 12757 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:34.927922 12894 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:35.093298 13060 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.094213 12832 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:35.094570 13075 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 13:15:35.094699 12894 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:35.094910 13060 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.011776 13045 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:35.080149 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:35.096241 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.096710 12119 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:35.083272 12122 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:15:35.097604 12122 pending_rounds.cc:85] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 2
I20260709 13:15:35.097911 12122 pending_rounds.cc:107] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:15:35.098311 11547 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:35.098277 12122 raft_consensus.cc:2917] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:15:35.098903 12122 raft_consensus.cc:2889] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:35.099673 12118 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:35.073191 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.100713 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.083662 12984 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:35.101397 13057 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:35.103406 13057 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.104440 12884 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.104642 13033 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.105465 11401 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:35.105588 11546 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:35.106135 13057 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:35.106390 13088 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:35.106878 13088 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.107303 13088 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.106734 13057 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:35.108358 13057 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:35.110854 13057 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:35.092356 13086 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 13:15:35.093477 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.078779 12883 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:35.016971 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 98036.4us
W20260709 13:15:35.102674 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.112064 12083 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:35.113796 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.114547 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:35.084348 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.115262 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.089686 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:35.064661 12837 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.097427 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.113217 12757 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.070914 13078 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:35.074764 11514 connection.cc:256] RPC call timeout handler was delayed by -1.2575375240000994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
I20260709 13:15:35.123593 11473 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:35.111245 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.158566 11545 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:35.125500 11547 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:35.159868 13053 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.159893 12995 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.161444 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.190757 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.192867 11396 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:35.196838 13026 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.196954 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.192646 12982 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.162226 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:35.163872 11469 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.199193 12927 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:35.199693 12927 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.197901 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.201037 12113 raft_consensus.cc:2410] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:35.201468 13088 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:35.163728 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 74111.2us
I20260709 13:15:35.164104 12883 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:35.164712 11542 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:35.204191 12883 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.204571 12960 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.204667 12883 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.207491 11394 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:35.209129 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:35.209879 11472 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:35.210626 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1320) took 1429 ms. Trace:
I20260709 13:15:35.220798 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.219950 11544 rpcz_store.cc:276] 0709 13:15:33.781116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.033973 (+252857us) service_pool.cc:224] Handling call
0709 13:15:34.045754 (+ 11781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:34.931672 (+885918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.931707 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:34.931766 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:34.931812 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.210407 (+278595us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54003e9ed4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:35.210451 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.210566 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21622,"raft.queue_time_us":36459,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":27575,"spinlock_wait_cycles":2015411456}
I20260709 13:15:35.222061 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:35.222594 11397 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.228287 13031 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:35.229333 13038 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.231657 12984 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.231792 12950 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.232653 13039 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:35.165592 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.234823 13053 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:35.235838 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:35.235981 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.237298 13053 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:35.121426 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:35.158608 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46628.6us
I20260709 13:15:35.155028 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.162016 12115 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:35.211848 12993 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:35.213006 13071 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.653s	user 0.002s	sys 0.001s
I20260709 13:15:35.213800 13055 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:35.214530 13064 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.215296 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:35.215618 12687 consensus_meta.cc:287] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.003s
I20260709 13:15:35.273494 13088 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:35.186712 12764 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:35.274610 12764 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.275059 12764 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:35.277350 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.277932 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.278254 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.219391 12874 consensus_meta.cc:287] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.000s
I20260709 13:15:35.279268 12874 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.216332 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:35.201895 11396 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:35.165329 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.282248 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.282742 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.244606 11399 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:35.118359 13057 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:35.191392 11514 connection.cc:256] RPC call timeout handler was delayed by -1.2476531779998368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
I20260709 13:15:35.273254 12687 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:35.237195 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:35.281088 11551 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:35.283308 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.290076 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:35.284004 11514 connection.cc:256] RPC call timeout handler was delayed by -1.2305375240000558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 13:15:35.286163 11546 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:15:35.290975 11546 pending_rounds.cc:85] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 2
I20260709 13:15:35.237877 13078 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:35.290915 11514 connection.cc:256] RPC call timeout handler was delayed by -1.2071375240000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:15:35.291363 11546 pending_rounds.cc:107] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:15:35.198217 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.086680 13044 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.306905 11546 raft_consensus.cc:2917] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
W20260709 13:15:35.307142 11514 leader_election.cc:336] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.502s (SENT)
I20260709 13:15:35.307821 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:35.293315 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:35.300191 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 63050.1us
I20260709 13:15:35.305636 11397 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:35.309942 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:35.235057 13052 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:35.323573 11546 raft_consensus.cc:2889] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:35.323693 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.271502 12923 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:35.329032 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.330442 11544 raft_consensus.cc:2393] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:35.332736 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 116498us
I20260709 13:15:35.327353 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.331532 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.336133 13064 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:15:35.337296 11514 leader_election.cc:336] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.532s (SENT)
I20260709 13:15:35.337759 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:35.310770 12757 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.345126 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.310810 13031 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:35.310942 12960 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:35.347265 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.302383 12995 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.349026 12112 raft_consensus.cc:2376] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3. Re-sending same reply.
I20260709 13:15:35.349519 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:35.351446 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:35.352296 13057 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.349800 13076 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.354946 12927 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.363282 13037 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:35.364504 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.346081 13063 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.346176 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.401700 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.374850 12757 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.349628 12894 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:15:35.411795 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.417586 12118 pending_rounds.cc:85] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 4
I20260709 13:15:35.417996 12118 pending_rounds.cc:107] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:15:35.407945 12883 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:35.340265 12923 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.383666 12960 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:35.413208 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.443231 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:35.573175 12122 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:35.460721 13026 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:35.326748 12837 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:15:35.371345 13031 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.574433 13031 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.574786 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.575989 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:35.576555 11400 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3. Re-sending same reply.
I20260709 13:15:35.577549 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:35.578245 11400 raft_consensus.cc:2368] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
W20260709 13:15:35.381589 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.579039 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:35.579447 12114 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:35.331907 13033 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:15:35.579728 11400 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
W20260709 13:15:35.539709 13051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.010s	sys 0.004s
W20260709 13:15:35.580298 13051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.010s	sys 0.005s
W20260709 13:15:35.400097 13058 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.008s	sys 0.010s
I20260709 13:15:35.418299 12118 raft_consensus.cc:2889] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:35.580664 13058 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.008s	sys 0.011s
W20260709 13:15:35.387198 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 310635us
I20260709 13:15:35.600052 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.165313 13026 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:36.166173 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.166640 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.164407 12883 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.146600 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.167833 13026 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.167887 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.168107 13051 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:35.679296 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.680356 13033 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:35.528182 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 189295us
I20260709 13:15:36.169092 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:35.676354 12837 consensus_queue.cc:1243] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:15:35.597501 13044 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:35.581560 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:36.195127 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0442375239999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
I20260709 13:15:35.671386 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:36.195827 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0094342689999394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 13:15:35.573304 12960 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.197341 13101 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:36.197341 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:36.156728 13037 consensus_meta.cc:287] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.000s
I20260709 13:15:35.458817 13053 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.171947 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:35.694248 12764 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:36.172220 12923 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:35.681074 13078 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.449143 13057 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.174450 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:35.669260 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.648044 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:35.570272 12757 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.213088 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.214620 11395 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:36.115268 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 940) took 1690 ms. Trace:
W20260709 13:15:35.814520 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.216050 11366 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:35.674226 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 942) took 1240 ms. Trace:
I20260709 13:15:35.572036 11401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:15:36.221143 11401 raft_consensus.cc:606] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:35.679857 12894 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.178731 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.179479 13058 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:36.194098 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 1.39913e+06us
W20260709 13:15:36.196208 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.184264 13051 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:35.569401 12961 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.202950 11399 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 4.
I20260709 13:15:36.203819 11542 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:36.233014 11551 pending_rounds.cc:85] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 1
I20260709 13:15:36.233439 11551 pending_rounds.cc:107] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:36.164785 12687 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:35.690934 13031 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:36.229594 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.198388 13033 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.207062 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.233726 11551 raft_consensus.cc:2889] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:36.208629 13053 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.233068 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.241706 13053 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.242350 13053 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:36.243860 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.245095 11543 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:36.245415 13051 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.209942 13078 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:36.245862 11473 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1366) took 1589 ms (client timeout 1657 ms). Trace:
I20260709 13:15:36.231058 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.246134 11474 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:15:36.246261 11473 rpcz_store.cc:269] 0709 13:15:34.656598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.681807 (+ 25209us) service_pool.cc:224] Handling call
0709 13:15:36.245680 (+1563873us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b540039bcd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:36.245793 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546919168}
I20260709 13:15:36.247045 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.224171 13030 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.248188 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.249490 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.250202 11395 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.223901 12923 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:36.251212 12923 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:36.246585 13078 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.251744 12923 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:36.252236 13078 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.256840 12764 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:36.257581 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.258131 11543 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:36.258901 11543 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.237548 12961 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.260017 11543 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.259948 12961 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:36.261943 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0749648890000572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31800000000000006s for a response
I20260709 13:15:36.235677 13063 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.226861 12894 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:36.263230 13063 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.264714 12894 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.227728 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:36.267345 12995 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.886s	user 0.007s	sys 0.004s
I20260709 13:15:36.242879 13053 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.274392 13053 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.276028 13053 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:36.221783 11393 rpcz_store.cc:276] 0709 13:15:34.425224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.885175 (+459951us) service_pool.cc:224] Handling call
0709 13:15:34.951216 (+ 66041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.111943 (+160727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.111978 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.112509 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.339147 (+226638us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:35.348286 (+  9139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.364293 (+ 16007us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540030af54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:35.364372 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.406716 (+ 42344us) raft_consensus.cc:1685] finished
0709 13:15:35.408152 (+  1436us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.665526 (+257374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369026944,"thread_start_us":212884,"threads_started":1,"wal-append.queue_time_us":287266}
I20260709 13:15:36.278934 11543 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:36.280069 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.281682 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.245335 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.283972 11401 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
W20260709 13:15:36.276538 13096 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.726s	user 0.006s	sys 0.018s
I20260709 13:15:36.235050 11400 raft_consensus.cc:2376] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2. Re-sending same reply.
I20260709 13:15:36.288232 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:36.288650 11399 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:36.289844 11397 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:36.291177 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.292832 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.293864 11393 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:36.295290 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:36.297070 13076 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:15:36.300029 13055 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:36.301406 13031 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:36.261936 12961 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.228403 12757 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:15:36.307888 12883 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:36.308355 11514 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.644s (SENT)
I20260709 13:15:36.218106 11398 rpcz_store.cc:276] 0709 13:15:34.424781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.878862 (+454081us) service_pool.cc:224] Handling call
0709 13:15:34.879271 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:34.911419 (+ 32148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.911455 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:34.924701 (+ 13246us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.230787 (+306086us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b18003e0800. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:35.387999 (+157212us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:35.388061 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.388159 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.114855 (+726696us) raft_consensus.cc:1685] finished
0709 13:15:36.114880 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.115172 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13159,"raft.queue_time_us":49375,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"spinlock_wait_cycles":17211136,"thread_start_us":156974,"threads_started":1,"wal-append.queue_time_us":1122478}
I20260709 13:15:36.310761 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.236315 13058 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.258932 13033 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:36.331260 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:36.332582 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:36.315212 13053 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.336577 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.335788 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:36.349269 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:36.346838 11398 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:36.357039 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:36.347388 11395 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:36.361994 13055 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:36.335348 13079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.014s	sys 0.035s
W20260709 13:15:36.362890 13079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.014s	sys 0.036s
I20260709 13:15:36.363624 13079 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:36.335564 12883 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:36.365191 12883 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:36.366554 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1342) took 1070 ms. Trace:
I20260709 13:15:36.366899 12757 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:36.367790 12757 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:36.367017 11550 rpcz_store.cc:276] 0709 13:15:35.296409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.366519 (+ 70110us) service_pool.cc:224] Handling call
0709 13:15:35.366863 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.373603 (+  6740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.373647 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.380664 (+  7017us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.697794 (+317130us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:35.697847 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.697941 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.366318 (+668377us) raft_consensus.cc:1685] finished
0709 13:15:36.366341 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.366497 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227240,"raft.queue_time_us":16436,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4850,"thread_start_us":248448,"threads_started":1,"wal-append.queue_time_us":931394}
I20260709 13:15:36.369639 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.370419 11550 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:36.371479 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.371626 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 945) took 1254 ms. Trace:
I20260709 13:15:36.372027 11394 rpcz_store.cc:276] 0709 13:15:35.117586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.118121 (+   535us) service_pool.cc:224] Handling call
0709 13:15:36.371436 (+1253315us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b54003a70d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:36.371568 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25533312}
I20260709 13:15:36.372872 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.373263 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:36.374024 11394 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:36.374054 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.375303 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.375936 11550 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:36.376397 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 369) took 1314 ms. Trace:
I20260709 13:15:36.376523 11394 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:36.376914 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1343) took 1080 ms. Trace:
I20260709 13:15:36.376955 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.378224 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.377390 11549 rpcz_store.cc:276] 0709 13:15:35.296929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.378154 (+ 81225us) service_pool.cc:224] Handling call
0709 13:15:35.378595 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.598164 (+219569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.598201 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.663946 (+ 65745us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.044142 (+380196us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.044199 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.044295 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.376706 (+332411us) raft_consensus.cc:1685] finished
0709 13:15:36.376726 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.376862 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25593,"raft.queue_time_us":212834,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":30700,"thread_start_us":183497,"threads_started":1,"wal-append.queue_time_us":587362}
I20260709 13:15:36.379338 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.379479 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.379632 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1344) took 1082 ms. Trace:
I20260709 13:15:36.380463 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.334081 12995 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:15:36.381153 11549 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 4.
I20260709 13:15:36.380398 11546 rpcz_store.cc:276] 0709 13:15:35.297272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.380997 (+ 83725us) service_pool.cc:224] Handling call
0709 13:15:35.381407 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.582099 (+200692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.582142 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.617755 (+ 35613us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.185309 (+567554us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.185348 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.185414 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.379440 (+194026us) raft_consensus.cc:1685] finished
0709 13:15:36.379460 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.379572 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189677,"raft.queue_time_us":224488,"spinlock_wait_cycles":34044800,"thread_start_us":394505,"threads_started":1,"wal-append.queue_time_us":712489}
I20260709 13:15:36.381955 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.382416 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.383857 11298 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 228) took 5879 ms. Trace:
I20260709 13:15:36.383934 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.384609 11546 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.382504 11549 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:36.377822 11394 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.380918 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1337) took 1096 ms. Trace:
I20260709 13:15:36.395237 11545 rpcz_store.cc:276] 0709 13:15:35.284643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.284788 (+   145us) service_pool.cc:224] Handling call
0709 13:15:35.294860 (+ 10072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.313064 (+ 18204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.330239 (+ 17175us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.342964 (+ 12725us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.549640 (+206676us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:35.549676 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.549742 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.380737 (+830995us) raft_consensus.cc:1685] finished
0709 13:15:36.380755 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.380864 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6475,"raft.queue_time_us":7057,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":31851,"thread_start_us":156627,"threads_started":1,"wal-append.queue_time_us":956818}
I20260709 13:15:36.388527 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.388782 11396 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.396891 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.389377 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.397855 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.398249 11399 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:36.398396 11545 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:36.390273 13053 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.390640 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:36.391557 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.391976 13091 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.384308 11298 rpcz_store.cc:276] 0709 13:15:30.504204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.504474 (+   270us) service_pool.cc:224] Handling call
0709 13:15:36.383800 (+5879326us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:31.266220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:31.266935 (+   715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:31.267023 (+    88us) write_op.cc:435] Acquired schema lock
0709 13:15:31.267048 (+    25us) tablet.cc:662] Decoding operations
0709 13:15:31.268147 (+  1099us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:31.268236 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:31.268266 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:15:31.269108 (+   842us) tablet.cc:701] Row locks acquired
0709 13:15:31.269158 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:15:31.269337 (+   179us) write_op.cc:270] Start()
0709 13:15:31.269491 (+   154us) write_op.cc:276] Timestamp: P: 1783602931269278 usec, L: 0
0709 13:15:31.269526 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:15:31.270201 (+   675us) log.cc:844] Serialized 3620 byte log entry
0709 13:15:32.453048 (+1182847us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b20001359f8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da94c7b91 00007f8da94e9a19 00007f8da94b48c3 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169 00007f8da94bd131 00007f8da94bc42d 000000000038f354
0709 13:15:33.913921 (+1460873us) op_driver.cc:464] REPLICATION: finished
0709 13:15:34.785993 (+872072us) write_op.cc:301] APPLY: starting
0709 13:15:34.786109 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:34.786386 (+   277us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:34.786431 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:35.337709 (+551278us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5c0027fd8c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dafb0b331 00007f8dafb0b221 00007f8dafb05f6c 00007f8dad26630b 00007f8dad265fd3 00007f8dad364e34 00007f8dad35097c 00007f8dad354d61 00007f8dad354d19 00007f8dad354ca9 00007f8dad354c71
0709 13:15:35.338311 (+   602us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:35.338709 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:15:35.338787 (+    78us) write_op.cc:312] APPLY: finished
0709 13:15:35.395253 (+ 56466us) log.cc:844] Serialized 71 byte log entry
0709 13:15:36.038147 (+642894us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.070143 (+ 31996us) write_op.cc:454] Released schema lock
0709 13:15:36.139287 (+ 69144us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:36.222316 (+ 83029us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.224525 (+  2209us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18058,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":871744,"apply.run_cpu_time_us":21700,"apply.run_wall_time_us":1420360,"num_ops":9,"prepare.queue_time_us":155293,"prepare.run_cpu_time_us":41366,"prepare.run_wall_time_us":175566,"raft.queue_time_us":1376801,"raft.run_cpu_time_us":21622,"raft.run_wall_time_us":324235,"replication_time_us":2585953,"spinlock_wait_cycles":31072384,"thread_start_us":2175539,"threads_started":5,"wal-append.queue_time_us":1185082,"wal-append.run_cpu_time_us":48558,"wal-append.run_wall_time_us":3283126}]]}
I20260709 13:15:36.392722 11546 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:36.393292 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1339) took 1106 ms. Trace:
I20260709 13:15:36.413080 11542 rpcz_store.cc:276] 0709 13:15:35.286976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.324691 (+ 37715us) service_pool.cc:224] Handling call
0709 13:15:35.344641 (+ 19950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:36.233233 (+888592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.393074 (+159841us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54002c65d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:36.393236 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974909184}
I20260709 13:15:36.393550 11550 raft_consensus.cc:2410] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:15:36.393410 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44269.2us
I20260709 13:15:36.369685 12764 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.394359 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41380 (request call id 112) took 1336 ms. Trace:
W20260709 13:15:36.386931 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 78622.5us
I20260709 13:15:36.376856 12112 rpcz_store.cc:276] 0709 13:15:35.062246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.065270 (+  3024us) service_pool.cc:224] Handling call
0709 13:15:36.376220 (+1310950us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54004c2554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:36.376334 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433313024}
I20260709 13:15:36.399639 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:36.402598 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:36.418031 11545 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:36.415650 11547 rpcz_store.cc:276] 0709 13:15:35.058203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.100499 (+ 42296us) service_pool.cc:224] Handling call
0709 13:15:36.394194 (+1293695us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b540041c3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:36.394291 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55369088}
I20260709 13:15:36.410542 11399 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.364082 13079 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.414306 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:36.415962 11514 leader_election.cc:336] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.674s (SENT)
I20260709 13:15:36.386879 13063 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:36.429981 11514 leader_election.cc:336] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.700s (SENT)
W20260709 13:15:36.435228 11514 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.881s (SENT)
I20260709 13:15:36.336043 11393 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:36.433629 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:36.436817 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:36.444722 11514 leader_election.cc:336] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.887s (SENT)
I20260709 13:15:36.415050 12997 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:36.418321 12764 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.434056 13079 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:36.433794 13065 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:36.412248 12923 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:36.461792 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:36.466274 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:36.473762 11550 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:36.464268 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:36.416108 13091 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:36.465200 12764 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.479143 11469 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:36.482401 11514 leader_election.cc:336] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.890s (SENT)
I20260709 13:15:36.471596 13113 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.474869 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.475047 13035 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:36.488121 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:36.483421 11514 leader_election.cc:336] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.001s (SENT)
I20260709 13:15:36.474821 13065 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:36.463301 12883 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:36.478838 13053 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:36.501242 11475 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.503211 13091 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.481305 11400 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:36.505244 12923 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.483189 13051 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.491554 13113 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.509905 11469 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:36.481144 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:36.537223 11472 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.548040 11472 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:36.536185 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.509485 13035 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:36.534165 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 303788us
W20260709 13:15:36.554050 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 68979.6us
I20260709 13:15:36.554735 11474 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:36.549875 12927 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:36.535650 12117 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:36.633541 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:36.615188 11514 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.049s (SENT)
I20260709 13:15:36.613507 11475 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:36.617982 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.632759 13115 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:15:36.633344 13123 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:36.616763 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 268326us
I20260709 13:15:36.535964 13053 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:36.635051 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.617408 13058 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:36.704682 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.705288 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.705621 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.705960 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.719992 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.650357 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 255) took 1259 ms. Trace:
I20260709 13:15:36.640247 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 340) took 1348 ms. Trace:
I20260709 13:15:36.756978 12123 rpcz_store.cc:276] 0709 13:15:35.391113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.647927 (+256814us) service_pool.cc:224] Handling call
0709 13:15:35.662813 (+ 14886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.663072 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.663102 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.174033 (+510931us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.504892 (+330859us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:36.504933 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.505000 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.650109 (+145109us) raft_consensus.cc:1685] finished
0709 13:15:36.650136 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.650294 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":330486,"threads_started":1,"wal-append.queue_time_us":378384}
I20260709 13:15:36.762217 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 943) took 2336 ms. Trace:
W20260709 13:15:36.647245 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.648370 11514 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.084s (SENT)
W20260709 13:15:36.787802 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.788300 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 139944us
W20260709 13:15:36.807510 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 390647us
I20260709 13:15:36.811662 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:36.816416 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:36.763331 11402 rpcz_store.cc:276] 0709 13:15:34.425450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.899336 (+473886us) service_pool.cc:224] Handling call
0709 13:15:34.926562 (+ 27226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.027949 (+101387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.027999 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.045789 (+ 17790us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.339830 (+294041us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:35.339886 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.276001 (+936115us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:36.761613 (+485612us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003a7fd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:36.761700 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.761803 (+   103us) raft_consensus.cc:1685] finished
0709 13:15:36.761817 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.762089 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264443648,"thread_start_us":183655,"threads_started":1,"wal-append.queue_time_us":384383}
I20260709 13:15:36.844156 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:36.847357 11514 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.137s (SENT)
I20260709 13:15:36.848214 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:36.854615 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:36.875711 11514 leader_election.cc:336] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.176s (SENT)
I20260709 13:15:36.875761 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:36.877552 11514 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.216s (SENT)
I20260709 13:15:36.878113 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:36.890908 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.891516 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.891589 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 275484us
I20260709 13:15:36.897133 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:36.902313 12123 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:36.792820 11474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:36.869856 13031 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.870589 12855 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:36.911950 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 500183us
W20260709 13:15:36.923640 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 173611us
I20260709 13:15:36.889600 13051 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:36.939383 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 255828us
I20260709 13:15:36.941210 11469 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:36.904534 13084 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.942030 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:36.943341 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.903585 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:36.909556 13063 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.951977 11475 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:36.896835 13101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.014s	sys 0.048s
I20260709 13:15:36.966920 12123 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:36.962191 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.912827 12874 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.909679 13078 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:36.970453 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:36.921123 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:36.645471 13065 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.965932 13084 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.907629 12997 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:36.929936 12855 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:36.969360 13101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.014s	sys 0.050s
I20260709 13:15:36.998417 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.989923 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:36.990713 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.939850 11474 raft_consensus.cc:606] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:36.761096 13035 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:36.970793 12927 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.967401 11472 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.969508 12995 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:37.001394 12123 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:37.001683 11477 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:37.006294 13053 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:37.010448 13127 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:15:36.973243 12883 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:36.976385 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.951401 13031 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.000986 11474 raft_consensus.cc:614] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:37.018105 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.012544 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:37.024663 11366 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:37.049101 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:36.997576 13084 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.050112 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:36.997905 12855 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.761550 12115 rpcz_store.cc:276] 0709 13:15:35.288595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.309540 (+ 20945us) service_pool.cc:224] Handling call
0709 13:15:36.510641 (+1201101us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4eee 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:36.533529 (+ 22888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:36.552739 (+ 19210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.554622 (+  1883us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:36.572336 (+ 17714us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:36.593755 (+ 21419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.612659 (+ 18904us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.637385 (+ 24726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13222144}
I20260709 13:15:36.998102 13101 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:37.011335 13131 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:15:37.014282 13125 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:15:37.051488 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 111397us
W20260709 13:15:37.014627 13120 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
W20260709 13:15:37.010948 12764 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.002s
I20260709 13:15:37.004743 13113 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:37.035423 13148 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:15:37.015843 12995 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.973240 13078 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.011425 12927 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:37.067261 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 174491us
I20260709 13:15:37.148604 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.196708 13146 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:37.196269 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 147283us
I20260709 13:15:37.067543 13035 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:37.093809 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.198936 11543 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:37.106734 11398 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:37.073608 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.271237 11395 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:37.201144 11471 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:37.173211 13103 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:15:37.263243 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 253) took 1919 ms. Trace:
I20260709 13:15:37.240147 13078 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.243502 12927 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:37.104303 13124 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:15:37.149394 11470 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:15:37.107313 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:37.210767 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.212126 11546 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:37.113979 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:37.066936 11394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.013s	sys 0.016s
W20260709 13:15:37.292382 11394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.017s	sys 0.016s
I20260709 13:15:37.203308 12997 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.202109 12764 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.202560 12995 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.197204 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:37.196344 11477 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:15:37.275415 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.300618 11401 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:37.272465 13044 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
W20260709 13:15:37.283566 13151 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:15:37.285804 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:37.277958 13142 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.075s	user 0.001s	sys 0.001s
I20260709 13:15:37.287621 11544 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:37.277007 13079 consensus_meta.cc:287] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.803s	user 0.004s	sys 0.006s
I20260709 13:15:37.293771 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1352) took 1007 ms. Trace:
I20260709 13:15:37.261904 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:37.299929 11476 raft_consensus.cc:2410] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:37.303896 11542 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:37.302903 11549 rpcz_store.cc:276] 0709 13:15:36.285910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.415962 (+130052us) service_pool.cc:224] Handling call
0709 13:15:36.416259 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.416571 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.416599 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.421428 (+  4829us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.758072 (+336644us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:36.758131 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.758226 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.293562 (+535336us) raft_consensus.cc:1685] finished
0709 13:15:37.293581 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.293715 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":434,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":20239,"thread_start_us":289992,"threads_started":1,"wal-append.queue_time_us":803952}
I20260709 13:15:37.301651 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:37.305086 11477 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:37.306119 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 957) took 1089 ms. Trace:
I20260709 13:15:37.306608 11401 rpcz_store.cc:276] 0709 13:15:36.216495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.284984 (+ 68489us) service_pool.cc:224] Handling call
0709 13:15:37.275229 (+990245us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:37.306044 (+ 30815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20923264}
I20260709 13:15:37.308099 13079 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.310061 12764 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:37.311058 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:37.311319 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.312075 11549 raft_consensus.cc:2410] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:37.312288 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.313836 11401 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:37.317076 12117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 259) took 1902 ms (client timeout 2134 ms). Trace:
W20260709 13:15:37.317549 12117 rpcz_store.cc:269] 0709 13:15:35.414475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.581240 (+166765us) service_pool.cc:224] Handling call
0709 13:15:37.316893 (+1735653us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400376ed4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:37.317023 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767406976}
I20260709 13:15:37.320751 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1350) took 1740 ms. Trace:
I20260709 13:15:37.321077 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1538) took 1907 ms. Trace:
I20260709 13:15:37.321480 11544 rpcz_store.cc:276] 0709 13:15:35.413915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.218021 (+804106us) service_pool.cc:224] Handling call
0709 13:15:36.228894 (+ 10873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.287451 (+1058557us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b5400266ea0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9890dc0 00007f8da991b1c0 00007f8da991c231 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c
0709 13:15:37.320823 (+ 33372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.321023 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20819840}
I20260709 13:15:37.327594 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:37.321121 11551 rpcz_store.cc:276] 0709 13:15:35.580547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.220679 (+640132us) service_pool.cc:224] Handling call
0709 13:15:36.232796 (+ 12117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.239797 (+  7001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.239836 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.240309 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.609793 (+369484us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.609836 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.609922 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.320540 (+710618us) raft_consensus.cc:1685] finished
0709 13:15:37.320558 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.320702 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":367637,"threads_started":1,"wal-append.queue_time_us":698458}
I20260709 13:15:37.328383 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1476) took 1140 ms. Trace:
I20260709 13:15:37.328711 11397 rpcz_store.cc:276] 0709 13:15:36.188153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.290531 (+102378us) service_pool.cc:224] Handling call
0709 13:15:37.327441 (+1036910us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:37.328332 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8289536}
W20260709 13:15:37.314785 13145 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.299s	user 0.004s	sys 0.000s
I20260709 13:15:37.313241 12855 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.332252 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 254) took 1960 ms. Trace:
I20260709 13:15:37.333424 12118 rpcz_store.cc:276] 0709 13:15:35.372213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.372360 (+   147us) service_pool.cc:224] Handling call
0709 13:15:35.417113 (+ 44753us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:35.417357 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.176856 (+759499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.176901 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.177491 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.415271 (+237780us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:36.415303 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.415365 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.327843 (+912478us) raft_consensus.cc:1685] finished
0709 13:15:37.327862 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.332176 (+  4314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6145280,"thread_start_us":237435,"threads_started":1,"wal-append.queue_time_us":377269}
W20260709 13:15:37.335351 13154 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:37.336086 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.336714 11551 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:37.299472 11472 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
W20260709 13:15:37.347391 13144 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
I20260709 13:15:37.354262 11401 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:37.354960 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1357) took 1041 ms. Trace:
I20260709 13:15:37.359994 11550 rpcz_store.cc:276] 0709 13:15:36.313361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.474536 (+161175us) service_pool.cc:224] Handling call
0709 13:15:36.476223 (+  1687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.476483 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.476519 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.478523 (+  2004us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.787927 (+309404us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.787990 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.788111 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.314126 (+526015us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:37.339280 (+ 25154us) raft_consensus.cc:1685] finished
0709 13:15:37.339313 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.354895 (+ 15582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42366848,"thread_start_us":298572,"threads_started":1,"wal-append.queue_time_us":537165}
I20260709 13:15:37.346197 11474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:37.370733 13101 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:37.374351 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 135) took 1142 ms. Trace:
I20260709 13:15:37.374845 11400 rpcz_store.cc:276] 0709 13:15:36.231883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.622775 (+390892us) service_pool.cc:224] Handling call
0709 13:15:36.632385 (+  9610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.927222 (+294837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.940370 (+ 13148us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:37.023537 (+ 83167us) log.cc:844] Serialized 26 byte log entry
0709 13:15:37.295123 (+271586us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:37.302032 (+  6909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.324768 (+ 22736us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.374093 (+ 49325us) raft_consensus.cc:1685] finished
0709 13:15:37.374121 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.374285 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":357294,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":13564,"thread_start_us":421460,"threads_started":2,"wal-append.queue_time_us":259152}
W20260709 13:15:37.380038 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.380484 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.380780 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.381034 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.385456 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:37.386101 12123 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:37.387844 11476 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 3. Ops: [1.3-1.3]
I20260709 13:15:37.388319 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.389595 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.384919 12764 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.395818 12764 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.398129 12764 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:37.279423 12113 rpcz_store.cc:276] 0709 13:15:35.343972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.346401 (+  2429us) service_pool.cc:224] Handling call
0709 13:15:36.207890 (+861489us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:36.217935 (+ 10045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.276779 (+ 58844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.276825 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.277465 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.639797 (+362332us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:36.639842 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.639912 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.199342 (+559430us) raft_consensus.cc:1685] finished
0709 13:15:37.201276 (+  1934us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.263156 (+ 61880us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40094,"raft.queue_time_us":41446,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"spinlock_wait_cycles":29479680,"thread_start_us":361878,"threads_started":1,"wal-append.queue_time_us":487496}
I20260709 13:15:37.400645 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1468) took 1823 ms. Trace:
I20260709 13:15:37.400727 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 960) took 1081 ms. Trace:
I20260709 13:15:37.401222 11402 rpcz_store.cc:276] 0709 13:15:36.318968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.881156 (+562188us) service_pool.cc:224] Handling call
0709 13:15:36.908889 (+ 27733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:36.909286 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:37.049630 (+140344us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.400396 (+350766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.400449 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:37.400494 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.400561 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.400673 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350514,"prepare.queue_time_us":199958,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"thread_start_us":109955,"threads_started":1}
I20260709 13:15:37.403905 13091 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:37.404994 12995 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.406291 12116 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:37.380086 11474 raft_consensus.cc:606] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
W20260709 13:15:37.407274 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 211259us
I20260709 13:15:37.394891 13078 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.408416 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.401129 11396 rpcz_store.cc:276] 0709 13:15:35.577382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.397374 (+819992us) service_pool.cc:224] Handling call
0709 13:15:36.414842 (+ 17468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.417632 (+  2790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.417670 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.445380 (+ 27710us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.869241 (+423861us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:37.049062 (+179821us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.400327 (+351265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.400402 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.400445 (+    43us) raft_consensus.cc:1685] finished
0709 13:15:37.400461 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.400589 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":351034,"prepare.queue_time_us":239845,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":217772,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":460108,"threads_started":2,"wal-append.queue_time_us":604833}
I20260709 13:15:37.411594 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.419821 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:37.421334 11401 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.421429 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1355) took 1134 ms. Trace:
I20260709 13:15:37.421978 11545 rpcz_store.cc:276] 0709 13:15:36.286665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.420673 (+134008us) service_pool.cc:224] Handling call
0709 13:15:36.421009 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:36.421482 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.421523 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:36.462630 (+ 41107us) log.cc:844] Serialized 52 byte log entry
0709 13:15:36.763847 (+301217us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:36.794233 (+ 30386us) log.cc:844] Serialized 14 byte log entry
0709 13:15:36.849125 (+ 54892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.849224 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.320453 (+471229us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180035a760. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:15:37.346724 (+ 26271us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:37.421152 (+ 74428us) raft_consensus.cc:1685] finished
0709 13:15:37.421183 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.421350 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57694,"prepare.queue_time_us":29968,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2523,"raft.queue_time_us":93479,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":31908,"spinlock_wait_cycles":28671744,"thread_start_us":282628,"threads_started":1,"wal-append.queue_time_us":455958}
I20260709 13:15:37.424165 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.425166 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.408304 13159 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:37.398579 13149 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:15:37.433943 13146 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.436264 13078 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:37.408242 13157 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:15:37.432783 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:37.435020 12083 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:37.440709 11402 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:37.407720 12112 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:37.431867 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 407257us
I20260709 13:15:37.384449 12927 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.407768 11474 raft_consensus.cc:614] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:37.466507 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:37.460353 13135 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.448s	user 0.001s	sys 0.001s
W20260709 13:15:37.460863 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.450641 11401 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:37.465368 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 966) took 1048 ms. Trace:
W20260709 13:15:37.468684 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.469333 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.468716 11399 rpcz_store.cc:276] 0709 13:15:36.417221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.272542 (+855321us) service_pool.cc:224] Handling call
0709 13:15:37.387370 (+114828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.387556 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:37.387813 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.465044 (+ 77231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.465093 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:37.465133 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.465199 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.465314 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38409,"prepare.queue_time_us":1025,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":38445,"threads_started":1,"wal-append.queue_time_us":38751}
W20260709 13:15:37.470615 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.471009 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.471359 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61871us
I20260709 13:15:37.454363 12764 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:37.456614 13078 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:37.480836 12118 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:37.469494 13159 raft_consensus.cc:993] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:37.484727 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:37.491421 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:37.468034 12995 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:37.498265 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:37.503050 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.505139 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 243389us
I20260709 13:15:37.506083 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:37.508893 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.509521 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.479142 12764 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.514959 13053 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:37.515570 13053 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.517817 13053 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:37.522583 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1354) took 1236 ms. Trace:
I20260709 13:15:37.523904 11547 rpcz_store.cc:276] 0709 13:15:36.286423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.419399 (+132976us) service_pool.cc:224] Handling call
0709 13:15:36.419765 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:36.434407 (+ 14642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:36.434436 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:36.434846 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.634439 (+199593us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.634486 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.634538 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.291027 (+656489us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b480002d9d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:15:37.420890 (+129863us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:37.522288 (+101398us) raft_consensus.cc:1685] finished
0709 13:15:37.522317 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.522508 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14205,"raft.queue_time_us":14317,"raft.run_cpu_time_us":9765,"raft.run_wall_time_us":854453,"spinlock_wait_cycles":47157504,"thread_start_us":169549,"threads_started":1,"wal-append.queue_time_us":528656}
I20260709 13:15:37.527097 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:37.528044 11547 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:37.500070 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:37.502097 12997 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:37.529935 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:37.532779 12997 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:37.609513 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 116664us
I20260709 13:15:37.576557 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:37.558208 11475 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:37.581732 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:37.563326 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.554880 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.613937 11547 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:37.562976 13030 raft_consensus.cc:1081] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:15:37.580885 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.580559 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:37.624001 11401 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.582520 12927 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:37.638799 11399 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:37.596578 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.523656 12923 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:37.513120 12764 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.613224 12997 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:37.643695 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.644447 12116 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:37.512919 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 134) took 1281 ms. Trace:
I20260709 13:15:37.495893 13078 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:37.560806 11474 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 4. Ops: []
I20260709 13:15:37.522262 13129 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.501579 12687 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:15:37.564445 12112 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:37.569585 13113 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:15:37.567818 13150 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.552s	user 0.003s	sys 0.000s
I20260709 13:15:37.649314 13055 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:37.605525 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 75672.1us
I20260709 13:15:37.611096 11551 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:37.622756 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:37.705096 11476 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:37.633500 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.705647 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:37.554668 13091 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.638046 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:37.520864 13084 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.647455 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:37.648238 11515 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:37.650266 11471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55518
W20260709 13:15:37.710341 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 62224.6us
I20260709 13:15:37.634558 13074 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:37.693444 13122 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.031s	sys 0.058s
W20260709 13:15:37.634445 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 111039us
I20260709 13:15:37.706564 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:37.710381 11471 raft_consensus.cc:606] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:37.575162 13130 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.573s	user 0.000s	sys 0.003s
I20260709 13:15:37.707104 13030 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:37.719594 11471 raft_consensus.cc:614] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:37.718379 11476 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:37.724334 13166 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.316s	user 0.002s	sys 0.000s
W20260709 13:15:37.792649 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.723033 13122 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.031s	sys 0.059s
W20260709 13:15:37.791033 13172 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:15:37.722451 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:37.789811 13181 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.115s	user 0.002s	sys 0.003s
W20260709 13:15:37.834730 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.770279 11545 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:37.773136 11469 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:37.713197 12923 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:37.709218 13055 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.768934 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:37.817240 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:37.769888 13137 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:37.721035 13074 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:37.813611 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:37.790794 13079 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:37.815099 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 164146us
I20260709 13:15:37.819533 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:37.713873 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:37.826265 11551 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:37.870733 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:37.871515 11551 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:37.645910 11394 rpcz_store.cc:276] 0709 13:15:36.231616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.468526 (+236910us) service_pool.cc:224] Handling call
0709 13:15:36.478391 (+  9865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.292788 (+814397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:37.292821 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:37.293338 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:15:37.383129 (+ 89791us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:37.440517 (+ 57388us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.478769 (+ 38252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.482525 (+  3756us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.495770 (+ 13245us) raft_consensus.cc:1685] finished
0709 13:15:37.500243 (+  4473us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.510968 (+ 10725us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91708,"prepare.queue_time_us":47188,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":211776,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":30166,"thread_start_us":881849,"threads_started":2,"wal-append.queue_time_us":144164}
I20260709 13:15:37.830739 13078 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:37.806434 13113 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:15:37.833318 11475 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:37.808915 13030 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:37.817618 13052 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:37.844447 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.851974 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 241573us
W20260709 13:15:37.768819 11514 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.456s (SENT)
I20260709 13:15:37.859318 11471 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.839241 12923 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.873158 11547 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 1.
I20260709 13:15:37.874944 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:37.900825 11476 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
W20260709 13:15:37.885658 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 116540us
W20260709 13:15:37.897239 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 391187us
W20260709 13:15:37.897305 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 128437us
I20260709 13:15:37.899518 13148 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:37.831950 12997 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:37.874310 11470 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:15:37.901024 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:37.901804 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.907511 11551 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.874603 13137 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:37.874464 12927 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:37.924734 11472 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:37.938143 11551 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
W20260709 13:15:37.939704 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 217314us
I20260709 13:15:37.904129 12923 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:37.921042 13176 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:15:37.951822 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 90901.5us
I20260709 13:15:37.952937 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:37.953544 11471 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
W20260709 13:15:37.957307 13133 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.994s	user 0.004s	sys 0.000s
I20260709 13:15:37.958578 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1341) took 2671 ms. Trace:
I20260709 13:15:37.958711 13079 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:37.959863 13079 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:37.958957 11548 rpcz_store.cc:276] 0709 13:15:35.287668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.357695 (+ 70027us) service_pool.cc:224] Handling call
0709 13:15:35.358059 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.372647 (+ 14588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.372694 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.373332 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.390905 (+1017573us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b180017c2c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:36.611561 (+220656us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:36.611600 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.611661 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.956674 (+1345013us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:37.958365 (+  1691us) raft_consensus.cc:1685] finished
0709 13:15:37.958388 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.958523 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207825,"raft.queue_time_us":23125,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7841,"spinlock_wait_cycles":26668672,"thread_start_us":220472,"threads_started":1,"wal-append.queue_time_us":1508590}
I20260709 13:15:37.931171 13113 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:37.947424 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.902344 13052 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:37.949451 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:37.949623 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:37.911118 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.960198 13079 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:37.945078 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:37.963397 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.919449 13084 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:37.967810 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:37.969988 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.970345 11551 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:37.970965 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1562) took 1435 ms. Trace:
I20260709 13:15:37.971303 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:37.971333 11551 rpcz_store.cc:276] 0709 13:15:36.535892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.967547 (+1431655us) service_pool.cc:224] Handling call
0709 13:15:37.970914 (+  3367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.972980 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:38.046276 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 75111.3us
W20260709 13:15:38.047922 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 75048.7us
W20260709 13:15:38.048799 13193 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:15:38.049104 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:38.050024 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:38.050854 11548 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:38.051715 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:38.054505 13167 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.651s	user 0.000s	sys 0.004s
I20260709 13:15:38.058701 13186 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.954326 12114 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:38.059322 13186 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:38.060684 13146 consensus_meta.cc:287] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.000s
I20260709 13:15:38.061082 13146 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.061470 13186 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:38.061918 13137 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.065517 12118 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.054183 13189 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.964963 13052 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.965238 13113 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:37.964385 13190 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:38.106943 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:37.920735 13030 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.072942 13044 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:38.109180 11468 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:37.960562 12764 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.103780 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:38.101979 12804 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:38.156642 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.157480 11468 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
I20260709 13:15:38.102567 12757 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:15:38.155751 13084 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.950275 13074 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.113943 12995 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:38.169299 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.168216 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1360) took 1846 ms. Trace:
I20260709 13:15:38.117693 12997 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:38.155445 11470 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:38.128971 13044 raft_consensus.cc:697] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:38.178253 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.161561 12116 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.197986 12119 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.179754 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.178725 11551 rpcz_store.cc:276] 0709 13:15:36.322170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.075165 (+1752995us) service_pool.cc:224] Handling call
0709 13:15:38.099638 (+ 24473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.099853 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:38.111723 (+ 11870us) log.cc:844] Serialized 14 byte log entry
0709 13:15:38.167226 (+ 55503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.167421 (+   195us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:38.167579 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.167735 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.168113 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3195,"prepare.queue_time_us":14001,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":13447,"thread_start_us":50954,"threads_started":1}
W20260709 13:15:38.183656 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.185712 11473 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:38.199085 11468 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:38.187026 12764 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:38.186560 13044 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.188194 13031 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:38.173157 13190 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:38.076607 13198 raft_consensus.cc:2771] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEARNER]: Leader pre-election decision while not in active config. Result: Term 4: won. RaftConfig: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:38.176415 13174 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:15:38.162001 13157 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 13:15:38.199600 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.114727 13079 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.164014 12804 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.203994 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.205435 13194 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:15:38.179250 13189 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:15:38.216413 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1361) took 1893 ms. Trace:
I20260709 13:15:38.216918 11551 rpcz_store.cc:276] 0709 13:15:36.322813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.212087 (+1889274us) service_pool.cc:224] Handling call
0709 13:15:38.215847 (+  3760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.216100 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.216126 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.216167 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:38.216195 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.216245 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.216354 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.175626 13113 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:38.206439 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 157396us
I20260709 13:15:38.223294 13079 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.231946 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.220472 13031 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.234565 11473 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:38.222203 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.210152 12923 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.222785 13190 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:38.239485 11473 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:38.240437 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.238713 13031 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.244521 13031 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:38.245508 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.256420 11468 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.256822 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.257577 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.274430 11473 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:38.236804 13113 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:38.283772 13187 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:15:38.284618 13149 raft_consensus.cc:1064] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:15:38.284919 11472 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:38.285238 13196 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.289984 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.312901 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.317942 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.283227 13030 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.314687 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.318746 11475 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:38.334406 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.335557 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.320096 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:38.319747 13149 raft_consensus.cc:1068] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Unable to promote non-voter b3abf4b5ee724203ad1226f5c46bfe46: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:15:38.348495 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 182694us
I20260709 13:15:38.344684 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.355765 11476 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:38.354655 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.306447 13196 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:38.368832 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.359951 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:38.371804 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.373148 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.364055 13212 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.377810 13212 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.378443 13055 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:38.389676 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.391979 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.392436 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.379397 13052 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.395607 11476 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:38.398808 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.398910 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:38.399713 11543 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.401055 13196 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.365170 12116 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:38.379987 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.373732 13182 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.380298 13030 raft_consensus.cc:2764] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:15:38.401975 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:38.410063 11549 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:38.410360 13177 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:38.410737 13177 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:38.411123 13177 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.411264 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:38.411875 11549 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.416404 13053 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.416867 13053 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:38.417765 13053 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.419339 12804 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:38.397941 13149 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.420974 11474 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.421416 11474 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:38.425155 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.372537 13190 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.406962 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.419339 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:38.408195 13173 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:38.423204 11514 leader_election.cc:336] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.029s (SENT)
I20260709 13:15:38.408748 12855 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:38.428972 12923 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:38.431845 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.430100 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41380 (request call id 115) took 1525 ms. Trace:
I20260709 13:15:38.432603 11468 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:15:38.432497 11545 rpcz_store.cc:276] 0709 13:15:36.904441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.424969 (+520528us) service_pool.cc:224] Handling call
0709 13:15:38.429925 (+1004956us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400404054. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:38.430044 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792048896}
I20260709 13:15:38.433648 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:38.430876 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.434278 12118 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 1.
W20260709 13:15:38.434521 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:38.426587 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.430855 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:38.436093 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.436435 13035 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:38.404520 13174 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.439831 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.441710 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.444793 11468 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:38.438906 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.442327 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:38.446410 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.430549 13053 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.447883 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.443948 13196 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.448928 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.447110 11469 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:15:38.456159 11468 raft_consensus.cc:2410] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:38.456821 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.457446 11469 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.458868 11515 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:38.459772 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:38.422309 11474 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:38.466730 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.468149 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.468750 11468 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:38.469559 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1363) took 2139 ms. Trace:
I20260709 13:15:38.441345 13189 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:38.464711 11550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930d965be7eb49448fa23456beb9925a"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:38.473635 11550 raft_consensus.cc:606] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:38.473999 11550 raft_consensus.cc:614] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:38.443378 12757 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:38.464779 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.464970 13182 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.429569 11470 raft_consensus.cc:2376] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3. Re-sending same reply.
I20260709 13:15:38.469974 11542 rpcz_store.cc:276] 0709 13:15:36.326290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.465210 (+2138920us) service_pool.cc:224] Handling call
0709 13:15:38.465516 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.465690 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.465714 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.465755 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:38.465784 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.465836 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.465938 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.431926 13055 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.471424 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1364) took 2139 ms. Trace:
I20260709 13:15:38.466739 12837 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:38.457413 11475 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:38.472817 11515 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.396499 13207 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:15:38.497439 11546 rpcz_store.cc:276] 0709 13:15:36.326905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.465878 (+2138973us) service_pool.cc:224] Handling call
0709 13:15:38.466132 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.466297 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.466323 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.466361 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:38.466391 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.466438 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.466527 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.500093 12083 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:38.500279 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.503602 11475 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:38.475037 11549 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
W20260709 13:15:38.475400 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 90022.1us
I20260709 13:15:38.449292 13212 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:38.492605 11550 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:38.508124 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1540) took 2137 ms. Trace:
I20260709 13:15:38.494309 11469 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:38.476420 12757 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.471334 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.506314 11543 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:38.466152 12883 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.496683 11474 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.507027 11549 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.509487 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:38.512032 11475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:38.512413 11542 raft_consensus.cc:1240] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:38.508664 11550 rpcz_store.cc:276] 0709 13:15:36.371093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.492117 (+2121024us) service_pool.cc:224] Handling call
0709 13:15:38.492418 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.507873 (+ 15455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.508065 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.523095 11544 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:38.524435 11475 raft_consensus.cc:606] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:38.524749 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1551) took 2151 ms. Trace:
I20260709 13:15:38.525090 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1559) took 2058 ms. Trace:
W20260709 13:15:38.523263 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 52036.3us
I20260709 13:15:38.525301 11549 rpcz_store.cc:276] 0709 13:15:36.373755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.506367 (+2132612us) service_pool.cc:224] Handling call
0709 13:15:38.506795 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.524322 (+ 17527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.524697 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.525491 11544 rpcz_store.cc:276] 0709 13:15:36.466863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.520192 (+2053329us) service_pool.cc:224] Handling call
0709 13:15:38.522907 (+  2715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.524876 (+  1969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.525041 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.524789 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1556) took 2149 ms. Trace:
I20260709 13:15:38.528285 12855 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.549878 12855 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:38.550000 11469 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.525344 12883 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:38.532243 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.551143 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.551175 12883 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:38.524955 11475 raft_consensus.cc:614] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:38.551795 11469 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.475821 13030 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:15:38.550901 12855 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.552826 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.553383 11475 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:38.554243 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.555124 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:38.536355 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:38.555969 13216 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:15:38.556701 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.556924 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:38.557453 11470 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:38.557596 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1362) took 2232 ms. Trace:
I20260709 13:15:38.558157 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.558261 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.521368 12757 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.559554 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.549659 11542 rpcz_store.cc:276] 0709 13:15:36.400082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.511946 (+2111864us) service_pool.cc:224] Handling call
0709 13:15:38.512247 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.524586 (+ 12339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.524746 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.554769 11475 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:38.474380 13149 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:15:38.551734 11468 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:38.558912 11471 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:15:38.539577 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:43418 (request call id 72) took 1061 ms. Trace:
I20260709 13:15:38.475207 13173 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:15:38.500960 13055 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.523182 11472 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3.
I20260709 13:15:38.507468 13203 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.499260 12837 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:38.597699 11469 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:38.517395 12116 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:38.551926 12883 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.517344 12923 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.557933 11551 rpcz_store.cc:276] 0709 13:15:36.325317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.220221 (+1894904us) service_pool.cc:224] Handling call
0709 13:15:38.220698 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.220977 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.221022 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.222471 (+  1449us) log.cc:844] Serialized 26 byte log entry
0709 13:15:38.399911 (+177440us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:38.399950 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.400016 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:38.557397 (+157381us) raft_consensus.cc:1685] finished
0709 13:15:38.557414 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.557543 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":160781,"threads_started":1,"wal-append.queue_time_us":261479}
I20260709 13:15:38.570784 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.561966 11546 rpcz_store.cc:276] 0709 13:15:37.483496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.538312 (+1054816us) service_pool.cc:224] Handling call
0709 13:15:38.539490 (+  1178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:38.560177 11473 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:38.577926 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1359) took 2257 ms. Trace:
I20260709 13:15:38.585927 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.544567 13177 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.609692 11550 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:15:38.726318 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.806871 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 128) took 1157 ms. Trace:
W20260709 13:15:38.807689 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.807520 11549 rpcz_store.cc:276] 0709 13:15:37.649309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.581423 (+932114us) service_pool.cc:224] Handling call
0709 13:15:38.581747 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.581945 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:38.583438 (+  1493us) log.cc:844] Serialized 14 byte log entry
0709 13:15:38.806588 (+223150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.806634 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:38.806672 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.806730 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.806830 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1116,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":221973,"threads_started":1}
I20260709 13:15:38.808912 11476 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:38.497603 13211 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.611069 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.578794 13207 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.810592 11468 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:38.609679 12837 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.610178 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 350) took 1045 ms. Trace:
I20260709 13:15:38.787362 13076 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.811741 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.839051 11471 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:38.840004 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.840082 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.840667 11468 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:38.841197 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.841820 11471 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:38.812309 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.612206 11297 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 3, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 3 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.813727 11548 rpcz_store.cc:276] 0709 13:15:36.320490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.052593 (+1732103us) service_pool.cc:224] Handling call
0709 13:15:38.052892 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.101393 (+ 48501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.101423 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.102083 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:15:38.220119 (+118036us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:38.220160 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.220222 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:38.555456 (+335234us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:38.560703 (+  5247us) raft_consensus.cc:1685] finished
0709 13:15:38.560724 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.577855 (+ 17131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52011,"raft.queue_time_us":113926,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":8331,"spinlock_wait_cycles":16356608,"thread_start_us":99780,"threads_started":1,"wal-append.queue_time_us":170594}
I20260709 13:15:38.844587 11297 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.588085 11470 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:15:38.787668 13229 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:15:38.822530 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.826243 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.823196 13211 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:38.826826 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.791706 13055 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:38.610118 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.829217 11473 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.851503 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:38.831533 13222 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:15:38.831231 13173 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.712917 13174 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:38.818986 13207 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:38.835738 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.846095 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.846741 11297 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.852960 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.838779 13076 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:38.864702 11297 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.866966 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.869207 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.869263 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.869956 11471 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:38.871189 13182 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.867620 11297 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.872319 13030 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:38.873009 13030 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:38.873864 13030 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:38.875300 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.876202 13149 raft_consensus.cc:1064] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
W20260709 13:15:38.876643 13149 raft_consensus.cc:1068] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Unable to promote non-voter b3abf4b5ee724203ad1226f5c46bfe46: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:15:38.876785 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.877552 13076 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.878130 11468 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:38.878297 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.878557 13076 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.879992 11297 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.857393 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.855118 13123 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:38.887313 11473 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:38.887355 11297 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.857813 13211 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:38.884140 13052 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:38.885371 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.893564 11477 raft_consensus.cc:2410] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:38.849622 12923 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:38.855268 13129 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.871848 13177 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:38.894106 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.898234 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.898820 11471 raft_consensus.cc:2376] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3. Re-sending same reply.
I20260709 13:15:38.931856 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:38.899734 11472 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:38.958295 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.958598 11365 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:38.856215 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:38.964332 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.895000 13174 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.964476 11469 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:38.823067 12114 rpcz_store.cc:276] 0709 13:15:37.564614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.570494 (+  5880us) service_pool.cc:224] Handling call
0709 13:15:37.570787 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.059321 (+488534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.609930 (+550609us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54004c1b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:38.610117 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839141120}
I20260709 13:15:38.969062 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:38.966006 13177 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:38.975394 13177 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.975510 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:38.975772 13177 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.960184 13186 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.981726 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:38.982506 12116 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:38.982599 13231 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:15:38.987013 11393 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:38.987052 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:38.987772 11395 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:38.988216 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 117) took 2080 ms. Trace:
I20260709 13:15:38.988234 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:38.989154 11395 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:38.989245 13149 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:38.988729 11550 rpcz_store.cc:276] 0709 13:15:36.907808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.528780 (+1620972us) service_pool.cc:224] Handling call
0709 13:15:38.529082 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.806511 (+277429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.988009 (+181498us) spinlock_profiling.cc:243] Waited 80.2 ms on lock 0x7b540041d7d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:38.988161 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184511232}
I20260709 13:15:38.992625 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:38.993933 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:38.989099 11402 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.996925 11550 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:15:38.997629 11550 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.998509 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:39.003122 12085 leader_election.cc:336] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Timed out: RequestConsensusVote RPC to 127.10.3.3:46781 timed out after 1.661s (SENT)
I20260709 13:15:39.004565 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:39.005426 13188 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.005712 13199 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:39.006829 11550 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.007474 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:39.009053 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:39.000797 12883 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.011262 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:39.020571 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:39.024546 13053 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:39.028750 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:38.973238 13123 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.035962 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:38.973477 13207 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:38.898524 12923 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.038481 11399 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:39.021808 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 265) took 1543 ms. Trace:
W20260709 13:15:39.081612 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:39.026296 12804 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:15:39.082948 13057 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:39.037955 13198 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:15:39.083379 13057 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:39.084231 13057 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:39.085119 12757 consensus_meta.cc:287] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.001s
I20260709 13:15:39.085408 13057 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.085644 12757 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.088231 12757 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:39.081239 12117 rpcz_store.cc:276] 0709 13:15:37.471891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.474110 (+  2219us) service_pool.cc:224] Handling call
0709 13:15:38.467717 (+993607us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4eee 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:38.582280 (+114563us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4f8c 00007f8da67f4ef6 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:38.582429 (+   149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.582641 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:38.582926 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 13:15:38.921560 (+338634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.930244 (+  8684us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:38.960726 (+ 30482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.967968 (+  7242us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.015017 (+ 47049us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":141,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":4978,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":35961,"spinlock_wait_cycles":25623296,"thread_start_us":281913,"threads_started":1,"wal-append.queue_time_us":385169}
I20260709 13:15:39.081513 11550 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.092556 12883 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:39.093961 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.094509 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.096604 13230 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:15:39.098392 13188 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.099529 11402 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:39.101066 11402 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:39.102766 13234 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:15:39.102774 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:39.104815 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.105379 11477 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.088275 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.113852 11476 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:39.101275 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:39.038162 12997 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:15:39.051527 11397 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:39.054015 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.145915 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.146463 11514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.545s (SENT) [suppressed 32 similar messages]
I20260709 13:15:39.146550 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:39.051478 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:39.147303 11402 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:39.147670 11402 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:39.147806 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 96408.9us
I20260709 13:15:39.051440 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:39.028630 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:39.034226 13074 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:39.052306 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.152931 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.013624 13160 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:15:39.150300 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.175115 11468 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.160436 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1374) took 1195 ms. Trace:
I20260709 13:15:39.175993 11545 rpcz_store.cc:276] 0709 13:15:37.964989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.862590 (+897601us) service_pool.cc:224] Handling call
0709 13:15:38.862968 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.866127 (+  3159us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:38.871066 (+  4939us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.160140 (+289074us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.160186 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.160223 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.160282 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.160383 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":144812,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":87,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":288775,"threads_started":1}
I20260709 13:15:39.173044 11477 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:15:39.143368 13224 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.594s	user 0.005s	sys 0.009s
I20260709 13:15:39.149111 13199 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:39.134541 13198 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:39.178658 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:39.181723 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 126) took 1551 ms. Trace:
I20260709 13:15:39.174021 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:39.180286 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.179829 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1370) took 1215 ms. Trace:
I20260709 13:15:39.186328 11545 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:39.177541 13079 consensus_queue.cc:1059] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:15:39.175426 11400 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.212792 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1376) took 1108 ms. Trace:
I20260709 13:15:39.188114 13177 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:39.214522 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.191980 12116 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:39.192823 13186 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.195123 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1368) took 1231 ms. Trace:
I20260709 13:15:39.094004 11550 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:39.152382 13074 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:39.197568 11547 rpcz_store.cc:276] 0709 13:15:37.630364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.539873 (+909509us) service_pool.cc:224] Handling call
0709 13:15:38.540209 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.540530 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.540557 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.540950 (+   393us) log.cc:844] Serialized 26 byte log entry
0709 13:15:38.874055 (+333105us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b18000c7f60. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:39.026482 (+152427us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:39.060725 (+ 34243us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.060979 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.061072 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.181382 (+120310us) raft_consensus.cc:1685] finished
0709 13:15:39.181497 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.181658 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33953,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":870,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":24819,"spinlock_wait_cycles":14531968,"thread_start_us":152228,"threads_started":1,"wal-append.queue_time_us":609415}
I20260709 13:15:39.132256 13192 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:15:39.213404 11545 rpcz_store.cc:276] 0709 13:15:38.104099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.177722 (+1073623us) service_pool.cc:224] Handling call
0709 13:15:39.186092 (+  8370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.212475 (+ 26383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.212718 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:39.218935 11544 rpcz_store.cc:276] 0709 13:15:37.963495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.788192 (+824697us) service_pool.cc:224] Handling call
0709 13:15:38.788578 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.789027 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.789069 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.789642 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.036204 (+246562us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.036242 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.036306 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.194878 (+158572us) raft_consensus.cc:1685] finished
0709 13:15:39.194900 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.195058 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5031,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7332,"thread_start_us":217243,"threads_started":1,"wal-append.queue_time_us":382481}
I20260709 13:15:39.288633 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:39.289440 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:39.289678 11547 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.290037 11544 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:39.238660 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.320048 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.323711 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43430 (request call id 1375) took 1258 ms. Trace:
I20260709 13:15:39.325897 11396 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:39.178788 12855 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 13:15:39.326763 11515 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:39.234197 11468 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:39.260633 13030 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:39.211766 13199 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:39.211472 13198 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:39.228897 13137 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:39.207127 11551 rpcz_store.cc:276] 0709 13:15:37.963995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.813384 (+849389us) service_pool.cc:224] Handling call
0709 13:15:38.816741 (+  3357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.816923 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:38.842742 (+ 25819us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.174799 (+332057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.174856 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.174897 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.174958 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.179756 (+  4798us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3061,"prepare.queue_time_us":37704,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":103,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":19534,"thread_start_us":331730,"threads_started":1}
W20260709 13:15:39.338934 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.341220 11547 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:39.346738 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.321141 13207 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:15:39.338265 13079 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:39.327158 11550 rpcz_store.cc:276] 0709 13:15:37.974436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.996795 (+1022359us) service_pool.cc:224] Handling call
0709 13:15:39.223989 (+227194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:39.356864 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.351915 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.370862 11365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:39.335541 13074 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.338722 13044 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
W20260709 13:15:39.387050 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.388151 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.392650 11468 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:39.372509 13030 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:39.393833 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.412777 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.413735 11477 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:39.381716 13079 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:39.375872 13137 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.423544 11395 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:39.425055 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.435848 11477 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:39.426417 13052 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:39.443216 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64932.1us
I20260709 13:15:39.427196 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1369) took 1463 ms. Trace:
I20260709 13:15:39.449142 11549 rpcz_store.cc:276] 0709 13:15:37.963746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.809070 (+845324us) service_pool.cc:224] Handling call
0709 13:15:38.823883 (+ 14813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.842924 (+ 19041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.842951 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.851277 (+  8326us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.159785 (+308508us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.159822 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.159903 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.426896 (+266993us) raft_consensus.cc:1685] finished
0709 13:15:39.426921 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.427125 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21520,"raft.queue_time_us":21791,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6903,"thread_start_us":308240,"threads_started":1,"wal-append.queue_time_us":523582}
I20260709 13:15:39.428886 13113 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:39.403146 13178 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:39.465497 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.467543 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.467725 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:39.468279 11477 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:39.424448 13199 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.472576 13148 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:39.484273 13245 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:39.487856 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.440163 13052 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:39.453994 13113 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:39.489590 11402 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.495676 11402 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.438697 13055 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:15:39.515579 11544 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:39.519649 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1371) took 1555 ms. Trace:
I20260709 13:15:39.522881 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1372) took 1558 ms. Trace:
I20260709 13:15:39.523527 11542 rpcz_store.cc:276] 0709 13:15:37.964484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.823866 (+859382us) service_pool.cc:224] Handling call
0709 13:15:38.853580 (+ 29714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.853905 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.853923 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.855533 (+  1610us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.151049 (+295516us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.151090 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.151154 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.522653 (+371499us) raft_consensus.cc:1685] finished
0709 13:15:39.522675 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.522819 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2339,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3054,"thread_start_us":295226,"threads_started":1,"wal-append.queue_time_us":528695}
I20260709 13:15:39.485036 13178 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:39.529460 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.531363 11477 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:39.521880 11546 rpcz_store.cc:276] 0709 13:15:37.964235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.817379 (+853144us) service_pool.cc:224] Handling call
0709 13:15:38.817641 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.818180 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.022823 (+204643us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.518732 (+495909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.518786 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.518828 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.518914 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.519596 (+   682us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24256,"prepare.queue_time_us":46028,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":1111,"raft.queue_time_us":362,"thread_start_us":663015,"threads_started":2}
I20260709 13:15:39.492275 13177 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:39.465090 13198 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.554831 13080 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:39.554198 11548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.011s	sys 0.020s
W20260709 13:15:39.584263 11548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.021s
W20260709 13:15:39.548243 12837 consensus_meta.cc:287] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.001s
I20260709 13:15:39.585105 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1373) took 1620 ms. Trace:
I20260709 13:15:39.558955 13113 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.559033 13052 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:39.560571 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.557619 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 411150us
I20260709 13:15:39.558058 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.566408 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.575927 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:39.587319 11546 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.585480 11548 rpcz_store.cc:276] 0709 13:15:37.964742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.844164 (+879422us) service_pool.cc:224] Handling call
0709 13:15:38.846300 (+  2136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.846488 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.014003 (+167515us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.584749 (+570746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.584805 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.584847 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.584928 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.585047 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8191,"prepare.queue_time_us":53516,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":725169,"threads_started":2}
I20260709 13:15:39.589133 11402 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:39.591742 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:39.592819 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.594784 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.606649 11514 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.638s (SENT)
W20260709 13:15:39.607843 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.608280 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.608659 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.608999 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.492061 13030 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.636284 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1398) took 1054 ms. Trace:
I20260709 13:15:39.637197 11476 rpcz_store.cc:276] 0709 13:15:38.582341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.132826 (+550485us) service_pool.cc:224] Handling call
0709 13:15:39.158616 (+ 25790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.158853 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.194119 (+ 35266us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.635762 (+441643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.635941 (+   179us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.635990 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.636065 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.636213 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36564,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1888,"thread_start_us":441213,"threads_started":1}
I20260709 13:15:39.643900 11546 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:39.636416 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.663823 11475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.023s
W20260709 13:15:39.664304 11475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.007s	sys 0.023s
I20260709 13:15:39.548388 13079 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.707896 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.647806 13113 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:39.725725 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.748557 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.748620 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.592984 12837 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.692858 13052 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.760320 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.763401 11477 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:39.691893 13207 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.593180 13080 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.697528 13053 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.765378 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.760604 13243 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:15:39.778527 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.748497 13190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.006s	sys 0.032s
I20260709 13:15:39.766161 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:39.785531 11402 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:39.781177 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.788906 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.789374 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.781739 11543 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:39.790333 11476 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:39.791675 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:39.792764 11477 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:39.765024 13055 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:39.791985 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1379) took 1564 ms. Trace:
I20260709 13:15:39.807446 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1377) took 1702 ms. Trace:
I20260709 13:15:39.833357 11545 rpcz_store.cc:276] 0709 13:15:38.105282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.323598 (+1218316us) service_pool.cc:224] Handling call
0709 13:15:39.348003 (+ 24405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.348206 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:39.359568 (+ 11362us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.807027 (+447459us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.807094 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.807147 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.807232 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.807378 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20364,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"thread_start_us":447100,"threads_started":1}
I20260709 13:15:39.804409 11550 rpcz_store.cc:276] 0709 13:15:38.227981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.379852 (+1151871us) service_pool.cc:224] Handling call
0709 13:15:39.380221 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.386728 (+  6507us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:39.386957 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.791690 (+404733us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.791739 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.791779 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.791839 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.791936 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":90,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"thread_start_us":404368,"threads_started":1}
W20260709 13:15:39.790488 13241 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:15:39.770678 13137 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:39.803332 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.849277 11477 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:15:39.794807 13190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.006s	sys 0.032s
I20260709 13:15:39.662154 13178 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.867704 11542 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:39.835371 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.836532 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:39.868777 11476 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:39.807855 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1378) took 1603 ms. Trace:
I20260709 13:15:39.869561 11551 rpcz_store.cc:276] 0709 13:15:38.204891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.374087 (+1169196us) service_pool.cc:224] Handling call
0709 13:15:39.375156 (+  1069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.375408 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.382913 (+  7505us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.807452 (+424539us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.807515 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.807568 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.807646 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.807786 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8853,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":10840,"raft.queue_time_us":59561,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":423765,"threads_started":1}
W20260709 13:15:39.856913 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.871567 13203 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:39.771116 13207 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.871994 13203 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:39.872835 13203 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:39.873986 13203 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.874405 13207 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:39.877158 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:39.877832 12117 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:39.874908 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1560) took 3396 ms. Trace:
I20260709 13:15:39.878968 13057 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:39.879376 13057 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:39.777931 13053 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.879367 11543 rpcz_store.cc:276] 0709 13:15:36.478883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.523738 (+2044855us) service_pool.cc:224] Handling call
0709 13:15:38.539737 (+ 15999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.791650 (+1251913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.873543 (+ 81893us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001351d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:39.874719 (+  1176us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b54001351d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:39.874854 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2882714880}
I20260709 13:15:39.896369 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:39.882157 13257 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.355s	user 0.001s	sys 0.001s
I20260709 13:15:39.838387 13055 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:39.915277 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:40.037776 11477 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:40.038679 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:39.925161 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.933631 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.039232 11477 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:39.880565 13057 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:39.977201 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:39.981748 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.986205 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1383) took 1679 ms. Trace:
I20260709 13:15:40.057597 11544 rpcz_store.cc:276] 0709 13:15:38.306530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.519764 (+1213234us) service_pool.cc:224] Handling call
0709 13:15:39.520087 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.520339 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.561194 (+ 40855us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.985808 (+424614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.985870 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.985925 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.986003 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.986140 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16576,"prepare.queue_time_us":40652,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":773,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":25011,"thread_start_us":423350,"threads_started":1}
I20260709 13:15:40.010355 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 130) took 1670 ms. Trace:
W20260709 13:15:40.059705 11514 leader_election.cc:336] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.032s (SENT)
I20260709 13:15:40.059980 11542 rpcz_store.cc:276] 0709 13:15:38.339779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.525595 (+1185816us) service_pool.cc:224] Handling call
0709 13:15:39.555601 (+ 30006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.868162 (+312561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.009999 (+141837us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54002c51d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:40.010268 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717247872}
W20260709 13:15:39.925161 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.012305 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 420636us
W20260709 13:15:39.979481 11547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.023s	sys 0.042s
W20260709 13:15:40.064145 11547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.023s	sys 0.042s
I20260709 13:15:40.066316 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1390) took 1055 ms. Trace:
I20260709 13:15:40.066747 11546 rpcz_store.cc:276] 0709 13:15:39.011314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.748308 (+736994us) service_pool.cc:224] Handling call
0709 13:15:39.778787 (+ 30479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.779021 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:39.801031 (+ 22010us) log.cc:844] Serialized 14 byte log entry
0709 13:15:40.065979 (+264948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.066033 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:40.066081 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.066146 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.066258 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195100,"prepare.queue_time_us":26327,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":232526,"threads_started":1}
I20260709 13:15:40.069695 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1384) took 1203 ms. Trace:
I20260709 13:15:40.069813 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1381) took 1831 ms. Trace:
W20260709 13:15:39.910450 13211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.008s	sys 0.033s
I20260709 13:15:40.060391 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:39.977319 11543 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 13:15:40.116801 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.008s	sys 0.005s
I20260709 13:15:40.132311 11543 pending_rounds.cc:85] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 1
I20260709 13:15:40.132665 11543 pending_rounds.cc:107] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:40.132946 11543 raft_consensus.cc:2889] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:40.134456 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:39.975050 13074 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
W20260709 13:15:40.037433 13129 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.140s	user 0.012s	sys 0.000s
I20260709 13:15:40.130489 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:40.161450 11544 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
W20260709 13:15:40.147522 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.157303 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97642.5us
W20260709 13:15:40.163439 11514 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.148s (SENT)
I20260709 13:15:40.141553 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:40.070317 11548 rpcz_store.cc:276] 0709 13:15:38.866734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.589954 (+723220us) service_pool.cc:224] Handling call
0709 13:15:39.590306 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.590570 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:39.650750 (+ 60180us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.719939 (+ 69189us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b180035aac0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:15:40.069244 (+349305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.069315 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:40.069373 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.069463 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.069618 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22543,"prepare.queue_time_us":59712,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":5760,"raft.queue_time_us":24927,"spinlock_wait_cycles":30254336,"thread_start_us":326463,"threads_started":1}
I20260709 13:15:40.164041 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:40.203159 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 226041us
I20260709 13:15:40.203774 13160 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:40.204161 13129 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:40.204312 13160 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:40.205369 13160 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:40.207300 13174 raft_consensus.cc:1064] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
W20260709 13:15:40.209343 11400 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.913s	user 0.007s	sys 0.001s
I20260709 13:15:40.209743 11400 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:40.210273 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 145) took 1239 ms (client timeout 1599 ms). Trace:
I20260709 13:15:40.282372 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:40.284686 11545 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:15:40.321517 11549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.021s	sys 0.002s
W20260709 13:15:40.322158 11549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.022s	sys 0.002s
I20260709 13:15:40.323134 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1382) took 2083 ms. Trace:
I20260709 13:15:40.323596 11549 rpcz_store.cc:276] 0709 13:15:38.239844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.478913 (+1239069us) service_pool.cc:224] Handling call
0709 13:15:39.499497 (+ 20584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.499689 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.521215 (+ 21526us) log.cc:844] Serialized 14 byte log entry
0709 13:15:40.322646 (+801431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.322722 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:40.322779 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.322870 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.323060 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21778,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":13566,"thread_start_us":800998,"threads_started":1}
W20260709 13:15:40.358390 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.391525 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 455609us
I20260709 13:15:40.279723 13192 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:40.209353 13174 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:40.203202 13190 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:40.235318 13074 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:40.495316 13271 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:40.669901 13080 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:40.670806 13271 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:40.673557 13271 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:40.562729 13055 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:40.577122 13258 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:40.678905 13052 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.913s	user 0.006s	sys 0.000s
I20260709 13:15:40.830922 13053 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:40.867547 13074 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:40.854898 13057 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.861837 13258 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.003439 13258 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:41.089560 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.090652 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.094103 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.094508 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.210639 11400 rpcz_store.cc:269] 0709 13:15:38.971226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.973046 (+  1820us) service_pool.cc:224] Handling call
0709 13:15:40.210211 (+1237165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.096302 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.259243 11394 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.892s	user 0.010s	sys 0.002s
I20260709 13:15:41.099651 11394 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:41.099668 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.101958 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1483) took 1204 ms. Trace:
I20260709 13:15:41.102571 11402 rpcz_store.cc:276] 0709 13:15:39.897250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.915484 (+ 18234us) service_pool.cc:224] Handling call
0709 13:15:41.101808 (+1186324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.103660 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.106257 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 900) took 1677 ms. Trace:
I20260709 13:15:41.106804 11477 rpcz_store.cc:276] 0709 13:15:39.429047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.102851 (+1673804us) service_pool.cc:224] Handling call
0709 13:15:41.106144 (+  3293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.108513 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.106231 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.108855 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.109508 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1484) took 1212 ms. Trace:
I20260709 13:15:41.109696 11477 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:41.109921 11402 rpcz_store.cc:276] 0709 13:15:39.897551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.105782 (+1208231us) service_pool.cc:224] Handling call
0709 13:15:41.109408 (+  3626us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.110325 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 921) took 1346 ms. Trace:
I20260709 13:15:41.098467 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.111614 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1485) took 1213 ms. Trace:
I20260709 13:15:41.112015 11400 rpcz_store.cc:276] 0709 13:15:39.897838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.097418 (+1199580us) service_pool.cc:224] Handling call
0709 13:15:41.111530 (+ 14112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.113262 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.114185 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1487) took 1138 ms. Trace:
I20260709 13:15:41.114571 11400 rpcz_store.cc:276] 0709 13:15:39.975975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.112805 (+1136830us) service_pool.cc:224] Handling call
0709 13:15:41.114102 (+  1297us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.117189 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 953536us
W20260709 13:15:41.118516 11514 leader_election.cc:336] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.227s (SENT)
I20260709 13:15:41.119264 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:41.127650 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.128365 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.134356 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.135784 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1486) took 1205 ms. Trace:
W20260709 13:15:41.137411 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.141126 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.141760 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.142191 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.142482 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.141498 11402 rpcz_store.cc:276] 0709 13:15:39.931033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.110685 (+1179652us) service_pool.cc:224] Handling call
0709 13:15:41.135671 (+ 24986us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.146466 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1400) took 1617 ms. Trace:
I20260709 13:15:41.147121 11545 rpcz_store.cc:276] 0709 13:15:39.528811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.143216 (+614405us) service_pool.cc:224] Handling call
0709 13:15:40.284269 (+141053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:40.285180 (+   911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.146134 (+860954us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.142946 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.149410 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.150955 11545 raft_consensus.cc:2410] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:41.152842 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:41.153486 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.163781 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.165973 11549 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:40.577869 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.183758 13052 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.184964 13052 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.185503 12837 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.110728 11477 rpcz_store.cc:276] 0709 13:15:39.763494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.108494 (+1345000us) service_pool.cc:224] Handling call
0709 13:15:41.110243 (+  1749us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.169729 11545 raft_consensus.cc:2410] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:41.171725 11551 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:15:41.172626 12114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.006s	sys 0.023s
W20260709 13:15:41.173378 11550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.003s	sys 0.007s
I20260709 13:15:41.166905 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 136) took 1124 ms. Trace:
I20260709 13:15:41.191428 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.192739 11477 raft_consensus.cc:2410] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:41.189049 11549 rpcz_store.cc:276] 0709 13:15:40.042021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.325115 (+283094us) service_pool.cc:224] Handling call
0709 13:15:41.164713 (+839598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.166577 (+  1864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.166843 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.193274 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 135) took 1151 ms. Trace:
W20260709 13:15:41.325218 11477 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 901) took 1895 ms (client timeout 2471 ms). Trace:
W20260709 13:15:41.325856 11477 rpcz_store.cc:269] 0709 13:15:39.429367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.190913 (+1761546us) service_pool.cc:224] Handling call
0709 13:15:41.325096 (+134183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.327310 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.328358 13177 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.329111 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.329339 13177 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.330611 11549 raft_consensus.cc:2410] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:41.331393 13207 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.332350 13207 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:41.323459 11551 rpcz_store.cc:276] 0709 13:15:40.041725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.293289 (+251564us) service_pool.cc:224] Handling call
0709 13:15:41.171004 (+877715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.188238 (+ 17234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.193214 (+  4976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.339564 13160 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:41.342110 13160 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.345438 13160 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:41.345873 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.346313 11550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.305s	user 0.004s	sys 0.008s
I20260709 13:15:41.185854 13052 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.357220 13265 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.358202 13265 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.360577 12923 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.361590 13080 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.361409 12923 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.361871 13284 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.362476 13080 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.363160 13284 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.364491 13285 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.365360 13285 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:41.365957 13284 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.368040 13080 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.372545 13286 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.373369 13286 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.368196 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.367364 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1403) took 1807 ms. Trace:
I20260709 13:15:41.376394 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.375330 11542 rpcz_store.cc:276] 0709 13:15:39.559907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.202105 (+642198us) service_pool.cc:224] Handling call
0709 13:15:40.202471 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:40.202661 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:41.143312 (+940651us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.366662 (+223350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.366751 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:41.366860 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.366951 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.367146 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169325,"prepare.queue_time_us":989000,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":175765,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":37743,"threads_started":1,"wal-append.queue_time_us":48124}
I20260709 13:15:41.377321 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:41.368865 13274 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.371701 13287 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:41.377362 11545 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 1.
I20260709 13:15:41.370442 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:41.394197 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47302.6us
W20260709 13:15:41.353000 12114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.284s	user 0.006s	sys 0.024s
W20260709 13:15:40.936064 13113 consensus_meta.cc:287] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.175s	user 0.012s	sys 0.000s
I20260709 13:15:41.340878 13207 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:41.429126 13303 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:15:41.430012 12923 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:41.431895 11514 leader_election.cc:336] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.551s (SENT)
I20260709 13:15:41.432027 13200 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:41.432318 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55195.9us
I20260709 13:15:41.433832 13285 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:41.434105 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:41.434818 13274 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.435614 13287 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:41.436275 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.437522 13285 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.437628 13274 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.440466 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1392) took 2266 ms. Trace:
I20260709 13:15:41.418071 13160 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.445060 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:41.450317 13250 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.651s	user 0.008s	sys 0.028s
I20260709 13:15:41.452207 13274 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.453172 11476 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:15:41.455542 13203 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.581s	user 0.016s	sys 0.000s
I20260709 13:15:41.457139 13177 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.446648 13207 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.457919 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.483904 11549 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:41.455543 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:41.459692 11514 leader_election.cc:336] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.579s (SENT)
I20260709 13:15:41.461756 13159 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.461735 13302 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:41.464798 13247 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.701s	user 0.008s	sys 0.013s
I20260709 13:15:41.475206 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.357241 13301 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.477972 11477 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:41.438095 13287 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:41.487804 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.492865 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:41.500414 11477 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 922) took 1730 ms (client timeout 2134 ms). Trace:
W20260709 13:15:41.500936 11477 rpcz_store.cc:269] 0709 13:15:39.769707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.326783 (+1557076us) service_pool.cc:224] Handling call
0709 13:15:41.498969 (+172186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.489503 11476 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 918) took 1782 ms (client timeout 2143 ms). Trace:
I20260709 13:15:41.482404 13274 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:41.502880 11476 rpcz_store.cc:269] 0709 13:15:39.700000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.369869 (+1669869us) service_pool.cc:224] Handling call
0709 13:15:41.482530 (+112661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.509713 11542 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:41.508435 11551 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:41.512218 11545 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:41.514547 11545 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:15:41.517366 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1390) took 2974 ms. Trace:
I20260709 13:15:41.519177 13301 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:41.521169 12923 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:41.521246 13277 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.716s	user 0.008s	sys 0.031s
I20260709 13:15:41.444603 11550 rpcz_store.cc:276] 0709 13:15:39.174494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.844789 (+670295us) service_pool.cc:224] Handling call
0709 13:15:39.845064 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.845295 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:40.041395 (+196100us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.347074 (+1305679us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.378104 (+ 31030us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:41.415352 (+ 37248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.415478 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.440397 (+ 24919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":955209,"prepare.queue_time_us":199304,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":17468,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":212,"spinlock_wait_cycles":127529216,"thread_start_us":1305167,"threads_started":1,"wal-append.queue_time_us":804290}
I20260709 13:15:41.522023 12923 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:40.070406 11547 rpcz_store.cc:276] 0709 13:15:38.238448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.440408 (+1201960us) service_pool.cc:224] Handling call
0709 13:15:39.440777 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.441034 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:39.475854 (+ 34820us) log.cc:844] Serialized 14 byte log entry
0709 13:15:40.069373 (+593519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.069444 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:40.069500 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.069600 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.069744 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4597,"prepare.queue_time_us":46692,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":7073,"thread_start_us":588404,"threads_started":1}
W20260709 13:15:41.531172 13057 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.006s
I20260709 13:15:41.533130 13203 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:41.535429 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.528055 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 140) took 1467 ms. Trace:
I20260709 13:15:41.508881 13274 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:41.534902 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75269.9us
I20260709 13:15:41.540591 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 916) took 1897 ms. Trace:
I20260709 13:15:41.542640 11477 rpcz_store.cc:276] 0709 13:15:39.641529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.526640 (+1885111us) service_pool.cc:224] Handling call
0709 13:15:41.538279 (+ 11639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.537432 11545 rpcz_store.cc:276] 0709 13:15:40.057993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.513260 (+1455267us) service_pool.cc:224] Handling call
0709 13:15:41.513995 (+   735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.524444 (+ 10449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.524743 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.517863 11473 rpcz_store.cc:276] 0709 13:15:38.543238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.888162 (+344924us) service_pool.cc:224] Handling call
0709 13:15:38.888452 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.888635 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.888659 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.889029 (+   370us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.349890 (+460861us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.349925 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.349986 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.374126 (+2024140us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.516655 (+142529us) raft_consensus.cc:1685] finished
0709 13:15:41.516681 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.517080 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602467072,"thread_start_us":460349,"threads_started":1,"wal-append.queue_time_us":774560}
I20260709 13:15:41.522038 13301 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.548362 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:41.549028 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 115226us
I20260709 13:15:41.505628 13302 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.555615 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1388) took 3024 ms. Trace:
I20260709 13:15:41.556293 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.549072 11476 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.557412 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 394) took 1861 ms. Trace:
I20260709 13:15:41.557484 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 907) took 2072 ms. Trace:
I20260709 13:15:41.558287 11473 rpcz_store.cc:276] 0709 13:15:39.485577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.555839 (+2070262us) service_pool.cc:224] Handling call
0709 13:15:41.557422 (+  1583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.560006 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.561371 11473 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:15:41.561851 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 917) took 1898 ms. Trace:
I20260709 13:15:41.562230 11473 rpcz_store.cc:276] 0709 13:15:39.663845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.559564 (+1895719us) service_pool.cc:224] Handling call
0709 13:15:41.561788 (+  2224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.511018 13160 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.567806 11549 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:41.569020 11542 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:41.568986 13159 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.570422 12923 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:41.569407 11514 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.718s (SENT)
I20260709 13:15:41.571159 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.571637 13159 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:41.572814 11514 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.794s (SENT)
I20260709 13:15:41.573350 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.573390 13287 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.553601 13302 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.575040 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 138) took 1529 ms. Trace:
I20260709 13:15:41.568835 13304 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:41.575542 11551 rpcz_store.cc:276] 0709 13:15:40.045721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.525246 (+1479525us) service_pool.cc:224] Handling call
0709 13:15:41.574064 (+ 48818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.574540 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.574583 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.574664 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:41.574739 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.574823 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.574959 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.575835 13304 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:41.578109 13286 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.580528 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 137) took 1526 ms. Trace:
I20260709 13:15:41.579105 13285 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.580938 11549 rpcz_store.cc:276] 0709 13:15:40.042301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.522671 (+1480370us) service_pool.cc:224] Handling call
0709 13:15:41.564856 (+ 42185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.568162 (+  3306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.568312 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.582517 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.536087 13207 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.583454 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 902) took 2153 ms. Trace:
W20260709 13:15:41.583809 13249 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.834s	user 0.016s	sys 0.018s
I20260709 13:15:41.585139 13207 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:41.587247 13256 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.548s	user 0.011s	sys 0.022s
W20260709 13:15:41.588788 11514 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.807s (SENT)
I20260709 13:15:41.590744 11476 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:41.583813 11477 rpcz_store.cc:276] 0709 13:15:39.429683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.546529 (+2116846us) service_pool.cc:224] Handling call
0709 13:15:41.583389 (+ 36860us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.556187 11474 rpcz_store.cc:276] 0709 13:15:38.542567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.881660 (+339093us) service_pool.cc:224] Handling call
0709 13:15:38.881978 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.882366 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.882400 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.882859 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.473024 (+590165us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.476978 (+  3954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.487359 (+ 10381us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.463654 (+1976295us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.554927 (+ 91273us) raft_consensus.cc:1685] finished
0709 13:15:41.554950 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.555379 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":329,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":572170624,"thread_start_us":513323,"threads_started":1,"wal-append.queue_time_us":766724}
I20260709 13:15:41.591365 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1414) took 2169 ms. Trace:
I20260709 13:15:41.593392 11476 rpcz_store.cc:276] 0709 13:15:39.421632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.547955 (+2126323us) service_pool.cc:224] Handling call
0709 13:15:41.591320 (+ 43365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.596414 12923 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.597421 12923 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.598459 12804 raft_consensus.cc:670] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:41.604265 13307 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.605286 13307 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.607095 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.607749 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.609282 13253 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.839s	user 0.009s	sys 0.019s
I20260709 13:15:41.610181 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1395) took 3063 ms. Trace:
I20260709 13:15:41.610592 11469 rpcz_store.cc:276] 0709 13:15:38.546187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.976374 (+430187us) service_pool.cc:224] Handling call
0709 13:15:38.976690 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.977244 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.977273 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.977677 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.383168 (+405491us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.392119 (+  8951us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.707104 (+314985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.708022 (+   918us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.582958 (+1874936us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.590405 (+  7447us) raft_consensus.cc:1685] finished
0709 13:15:41.595768 (+  5363us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.609998 (+ 14230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19443,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":87789,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6111,"spinlock_wait_cycles":655958912,"thread_start_us":719852,"threads_started":2,"wal-append.queue_time_us":618179}
W20260709 13:15:41.613032 13297 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.775s	user 0.006s	sys 0.007s
W20260709 13:15:41.613487 13279 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.780s	user 0.058s	sys 0.064s
W20260709 13:15:41.615072 13254 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.849s	user 0.016s	sys 0.017s
I20260709 13:15:41.601894 13200 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.617398 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1392) took 3071 ms. Trace:
I20260709 13:15:41.617251 13200 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.618427 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1402) took 2805 ms. Trace:
W20260709 13:15:41.619596 13272 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.950s	user 0.023s	sys 0.054s
I20260709 13:15:41.618930 11476 rpcz_store.cc:276] 0709 13:15:38.812979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.599875 (+2786896us) service_pool.cc:224] Handling call
0709 13:15:41.617721 (+ 17846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.618079 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.618108 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.618155 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:41.618188 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.618246 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.618351 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.592890 13304 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:41.623199 12804 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.623286 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.621596 11545 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:41.600135 13310 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:41.632560 13055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.000s	sys 0.019s
I20260709 13:15:41.601847 13302 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:41.639086 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.640422 13055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.001s	sys 0.019s
W20260709 13:15:41.645490 13281 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.976s	user 0.034s	sys 0.048s
I20260709 13:15:41.639416 13310 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.648382 12923 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.638543 13301 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:41.642299 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.654611 13246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.959s	user 0.005s	sys 0.051s
I20260709 13:15:41.639313 12804 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.660102 13246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.972s	user 0.006s	sys 0.052s
I20260709 13:15:41.655771 12117 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:41.651569 13159 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:41.666326 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.670038 12804 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.671645 13200 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:41.665609 13282 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.812s	user 0.001s	sys 0.015s
W20260709 13:15:41.665875 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 70296.1us
I20260709 13:15:41.676093 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 153) took 2856 ms. Trace:
I20260709 13:15:41.676225 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.676249 13269 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.859s	user 0.060s	sys 0.106s
I20260709 13:15:41.677903 12113 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:41.678161 11548 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:41.678453 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 148) took 2653 ms. Trace:
I20260709 13:15:41.671583 13173 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.617875 11470 rpcz_store.cc:276] 0709 13:15:38.545418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.979567 (+434149us) service_pool.cc:224] Handling call
0709 13:15:39.011696 (+ 32129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.011922 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.011949 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.012444 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.420820 (+408376us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.490838 (+ 70018us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.491075 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.496268 (+  5193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.614175 (+2117907us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.616832 (+  2657us) raft_consensus.cc:1685] finished
0709 13:15:41.616988 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.617223 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21827,"prepare.queue_time_us":47202,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":40886,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":16961,"spinlock_wait_cycles":662813056,"thread_start_us":392491,"threads_started":1,"wal-append.queue_time_us":623632}
I20260709 13:15:41.679769 13173 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:41.665537 13263 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.798s	user 0.035s	sys 0.044s
I20260709 13:15:41.683732 13160 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.679785 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 155) took 2528 ms. Trace:
I20260709 13:15:41.690325 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 985) took 2570 ms. Trace:
I20260709 13:15:41.686174 11397 rpcz_store.cc:276] 0709 13:15:39.151498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.300117 (+148619us) service_pool.cc:224] Handling call
0709 13:15:39.312232 (+ 12115us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:39.312694 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.312734 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:39.329001 (+ 16267us) log.cc:844] Serialized 52 byte log entry
0709 13:15:39.557019 (+228018us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:39.557076 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.557177 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.674164 (+2116987us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.679301 (+  5137us) raft_consensus.cc:1685] finished
0709 13:15:41.679325 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.679600 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7180,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":4692,"spinlock_wait_cycles":402036480,"thread_start_us":202520,"threads_started":1,"wal-append.queue_time_us":418384}
I20260709 13:15:41.690739 11399 rpcz_store.cc:276] 0709 13:15:39.119648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.154504 (+ 34856us) service_pool.cc:224] Handling call
0709 13:15:39.176450 (+ 21946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.176716 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.176754 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.186561 (+  9807us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.642139 (+455578us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.642192 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.642287 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.664483 (+2022196us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.684632 (+ 20149us) raft_consensus.cc:1685] finished
0709 13:15:41.684653 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.690145 (+  5492us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338866944,"thread_start_us":455169,"threads_started":1,"wal-append.queue_time_us":680158}
I20260709 13:15:41.691845 13265 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.695982 13310 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.677423 11474 rpcz_store.cc:276] 0709 13:15:38.820057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.625677 (+2805620us) service_pool.cc:224] Handling call
0709 13:15:41.652810 (+ 27133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.675462 (+ 22652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.675521 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.675649 (+   128us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:41.675692 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.675817 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.675984 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.696606 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 107872us
W20260709 13:15:41.697337 11514 leader_election.cc:336] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.860s (SENT)
I20260709 13:15:41.699501 13301 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.702227 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.660626 13302 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.678891 11401 rpcz_store.cc:276] 0709 13:15:38.985513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.985639 (+   126us) service_pool.cc:224] Handling call
0709 13:15:38.985923 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.051065 (+ 65142us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:39.522079 (+471014us) log.cc:844] Serialized 14 byte log entry
0709 13:15:40.202446 (+680367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.202493 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:40.202530 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.602530 (+1400000us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b54003d31d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:41.622003 (+ 19473us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.634158 (+ 12155us) inbound_call.cc:177] Queueing success response
0709 13:15:41.648959 (+ 14801us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
Metrics: {"mutex_wait_us":223007,"prepare.queue_time_us":29399,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2646,"raft.queue_time_us":269433,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":272353920,"thread_start_us":857479,"threads_started":2,"wal-append.queue_time_us":1331044}
I20260709 13:15:41.707597 13302 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.694896 12113 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:15:41.694875 13295 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.796s	user 0.003s	sys 0.013s
I20260709 13:15:41.557971 12114 rpcz_store.cc:276] 0709 13:15:39.695558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.697025 (+  1467us) service_pool.cc:224] Handling call
0709 13:15:39.765779 (+ 68754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.794272 (+ 28493us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:40.033280 (+239008us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.507648 (+1474368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.538904 (+ 31256us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:41.556894 (+ 17990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.557026 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.557280 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6946,"prepare.queue_time_us":480855,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":3587,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":35740,"spinlock_wait_cycles":284711680,"thread_start_us":1565571,"threads_started":2,"wal-append.queue_time_us":803933}
I20260709 13:15:41.723423 12804 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.724023 12804 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.727191 13055 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.727843 13055 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:41.729049 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.729729 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.730505 13055 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:41.711575 13265 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.685943 13159 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.739593 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.744402 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.745471 12119 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:41.711230 13301 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.711102 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.710920 13274 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.753098 13159 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.760452 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.760543 11474 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:41.760838 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:41.760592 13264 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.889s	user 0.019s	sys 0.070s
I20260709 13:15:41.761118 12117 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:15:41.763075 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.774034 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.774391 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.774680 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.775130 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.775910 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:41.776394 13055 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.777032 13055 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.778201 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1401) took 3194 ms. Trace:
I20260709 13:15:41.776211 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.779795 11474 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:41.781345 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.782050 11474 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:41.783270 13055 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.787125 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.763173 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.764556 13252 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 2.074s	user 0.008s	sys 0.016s
I20260709 13:15:41.766470 12804 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.754679 13173 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:41.797030 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.798379 13113 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:41.770011 13268 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.566s	user 0.013s	sys 0.022s
I20260709 13:15:41.798872 13113 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.799263 13113 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.801119 11393 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:41.802088 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.803350 13173 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:41.803476 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:41.804023 13173 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.804478 13173 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:41.806686 13267 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.013s	user 0.004s	sys 0.017s
I20260709 13:15:41.802807 11474 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:15:41.803323 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44807.2us
W20260709 13:15:41.808089 13288 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.890s	user 0.008s	sys 0.011s
I20260709 13:15:41.809852 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:41.810418 13292 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.960s	user 0.002s	sys 0.012s
I20260709 13:15:41.788260 13265 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:15:41.811483 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 986) took 2691 ms. Trace:
W20260709 13:15:41.812306 13291 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.018s	user 0.003s	sys 0.016s
I20260709 13:15:41.812834 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1397) took 3234 ms. Trace:
I20260709 13:15:41.813236 11471 rpcz_store.cc:276] 0709 13:15:38.578750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.037122 (+458372us) service_pool.cc:224] Handling call
0709 13:15:39.068691 (+ 31569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.489057 (+420366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.489093 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.522945 (+ 33852us) log.cc:844] Serialized 26 byte log entry
0709 13:15:40.199920 (+676975us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:41.109717 (+909797us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.110383 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.110539 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.807261 (+696722us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.812397 (+  5136us) raft_consensus.cc:1685] finished
0709 13:15:41.812504 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.812628 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":983392,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":66,"raft.queue_time_us":604257,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":99779,"spinlock_wait_cycles":31575808,"thread_start_us":957239,"threads_started":2,"wal-append.queue_time_us":1365760}
I20260709 13:15:41.771572 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:41.815301 12118 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:41.773046 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:41.778571 11477 rpcz_store.cc:276] 0709 13:15:38.584147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.593390 (+3009243us) service_pool.cc:224] Handling call
0709 13:15:41.632487 (+ 39097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.656345 (+ 23858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.656366 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.673248 (+ 16882us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.755414 (+ 82166us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:41.755443 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.755489 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.777993 (+ 22504us) raft_consensus.cc:1685] finished
0709 13:15:41.778010 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.778127 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2017,"raft.queue_time_us":5449,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":1298,"thread_start_us":51747,"threads_started":1,"wal-append.queue_time_us":89080}
I20260709 13:15:41.764118 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.778718 12119 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:15:41.794981 13199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.132s	user 0.027s	sys 0.069s
I20260709 13:15:41.793552 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:41.817373 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 120094us
I20260709 13:15:41.818745 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.819907 13286 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:41.820185 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.820845 13287 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:41.821286 13287 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.821607 13287 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.820353 13286 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.827703 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:41.827910 12118 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:41.829548 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.830125 12119 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:41.819432 13052 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:41.831233 11514 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.947s (SENT)
I20260709 13:15:41.831249 13076 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:41.831848 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.831861 13076 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:41.833253 11514 leader_election.cc:336] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.953s (SENT)
I20260709 13:15:41.833076 13076 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.833688 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:41.834460 13076 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.834481 13265 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.835402 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.836155 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.836207 12112 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:41.836671 13030 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:41.811941 11398 rpcz_store.cc:276] 0709 13:15:39.119886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.230239 (+110353us) service_pool.cc:224] Handling call
0709 13:15:39.230635 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.230894 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.230932 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.320269 (+ 89337us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.781437 (+461168us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.781470 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.781530 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.805847 (+2024317us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.811048 (+  5201us) raft_consensus.cc:1685] finished
0709 13:15:41.811067 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.811321 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172401408,"thread_start_us":430618,"threads_started":1,"wal-append.queue_time_us":1444723}
I20260709 13:15:41.837905 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.835492 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:41.838590 12122 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:41.814695 13055 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:15:41.841658 13055 consensus_queue.cc:1243] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:41.841866 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.830914 13188 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:41.843451 13188 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:41.843873 13188 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.836337 13160 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:41.766858 12883 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:15:41.838591 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:41.817602 13199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.169s	user 0.028s	sys 0.069s
W20260709 13:15:41.848418 11514 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.988s (SENT)
W20260709 13:15:41.853274 13251 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 2.207s	user 0.009s	sys 0.015s
I20260709 13:15:41.853374 13300 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.853883 13300 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:41.854332 13300 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:41.854748 13266 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.025s	user 0.007s	sys 0.011s
W20260709 13:15:41.856706 13261 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.402s	user 0.013s	sys 0.019s
W20260709 13:15:41.859174 13270 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.169s	user 0.008s	sys 0.012s
I20260709 13:15:41.859607 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1396) took 3296 ms. Trace:
I20260709 13:15:41.859995 11475 rpcz_store.cc:276] 0709 13:15:38.562908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.020565 (+457657us) service_pool.cc:224] Handling call
0709 13:15:39.043855 (+ 23290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.044171 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.044195 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.044846 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.664674 (+619828us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.752397 (+ 87723us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.752876 (+   479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.753017 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.876527 (+123510us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800218e00. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c557 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:15:41.855880 (+1979353us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.859264 (+  3384us) raft_consensus.cc:1685] finished
0709 13:15:41.859281 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.859461 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83841,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":377133,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":27674,"spinlock_wait_cycles":173988480,"thread_start_us":532684,"threads_started":1,"wal-append.queue_time_us":938241}
I20260709 13:15:41.862406 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.864637 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.828981 13286 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:41.828411 13198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.007s	sys 0.012s
I20260709 13:15:41.868372 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.835417 13265 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:41.865672 11475 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:41.871230 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.873272 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.874516 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.828025 12114 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:41.849327 13160 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:41.878371 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.879331 11394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 146) took 2907 ms (client timeout 1808 ms). Trace:
I20260709 13:15:41.879443 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.872367 13198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.007s	sys 0.013s
W20260709 13:15:41.880249 13283 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.187s	user 0.009s	sys 0.013s
I20260709 13:15:41.881695 12119 raft_consensus.cc:2410] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:15:41.885773 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.888697 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 154) took 3068 ms. Trace:
I20260709 13:15:41.889030 11469 rpcz_store.cc:276] 0709 13:15:38.820787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.864537 (+3043750us) service_pool.cc:224] Handling call
0709 13:15:41.877495 (+ 12958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.888339 (+ 10844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.888366 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.888409 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:41.888448 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.888509 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.888644 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:41.880695 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:41.892139 11515 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:41.892390 13260 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 2.049s	user 0.011s	sys 0.023s
I20260709 13:15:41.883324 13160 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.883406 13030 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:41.897123 13248 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 2.023s	user 0.026s	sys 0.006s
W20260709 13:15:41.897436 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 62488.6us
I20260709 13:15:41.895280 12118 pending_rounds.cc:85] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 4
I20260709 13:15:41.890410 13080 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:41.911459 13278 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.089s	user 0.004s	sys 0.014s
W20260709 13:15:41.882728 11394 rpcz_store.cc:269] 0709 13:15:38.971506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.975244 (+  3738us) service_pool.cc:224] Handling call
0709 13:15:41.879074 (+2903830us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540030aa54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:41.879241 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792609536}
I20260709 13:15:41.913335 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 157) took 2467 ms. Trace:
I20260709 13:15:41.893738 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.913739 11395 rpcz_store.cc:276] 0709 13:15:39.445753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.446703 (+   950us) service_pool.cc:224] Handling call
0709 13:15:39.447199 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.468979 (+ 21780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.469011 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.476579 (+  7568us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.813643 (+337064us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:39.813687 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.813766 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.910586 (+2096820us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.912814 (+  2228us) raft_consensus.cc:1685] finished
0709 13:15:41.912845 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.913142 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35033,"raft.queue_time_us":25639,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7354,"spinlock_wait_cycles":75027200,"thread_start_us":329601,"threads_started":1,"wal-append.queue_time_us":1311137}
I20260709 13:15:41.898793 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1402) took 2347 ms. Trace:
I20260709 13:15:41.915570 11474 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:41.915906 11548 rpcz_store.cc:276] 0709 13:15:39.551814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.164675 (+612861us) service_pool.cc:224] Handling call
0709 13:15:40.164988 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.678634 (+1513646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.898522 (+219888us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540004a454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:41.898731 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3473107840}
I20260709 13:15:41.916545 13129 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:41.899436 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.901326 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.901463 11393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 152) took 2913 ms (client timeout 2049 ms). Trace:
I20260709 13:15:41.918080 12112 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
W20260709 13:15:41.918203 11393 rpcz_store.cc:269] 0709 13:15:38.987861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.987991 (+   130us) service_pool.cc:224] Handling call
0709 13:15:41.901245 (+2913254us) spinlock_profiling.cc:243] Waited 2.78 s on lock 0x7b5400232154. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:41.901402 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6389602176}
I20260709 13:15:41.902473 11396 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:41.900336 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.905122 13265 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:41.905727 13312 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.194s	user 0.002s	sys 0.000s
I20260709 13:15:41.920356 13265 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:41.905900 13289 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.075s	user 0.015s	sys 0.002s
I20260709 13:15:41.905998 13274 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:41.907284 13275 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.452s	user 0.009s	sys 0.014s
I20260709 13:15:41.903684 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.916831 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:41.919886 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.882372 13307 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:41.921687 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.920611 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1394) took 3374 ms. Trace:
I20260709 13:15:41.921265 11471 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:41.937103 11548 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:41.923704 13265 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:41.927874 11514 leader_election.cc:336] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.081s (SENT)
I20260709 13:15:41.908363 13030 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.939869 11469 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:41.940667 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.941555 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.936848 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:41.931280 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.942358 12122 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:41.909772 12118 pending_rounds.cc:107] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:15:41.943375 12118 raft_consensus.cc:2889] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:41.944335 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:41.942378 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:41.945602 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:41.945338 11477 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:41.947413 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.951239 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.952235 13302 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.946089 13030 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.961772 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.945199 13307 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:41.965106 13305 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.965719 13178 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.936679 11472 rpcz_store.cc:276] 0709 13:15:38.545932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.975779 (+429847us) service_pool.cc:224] Handling call
0709 13:15:38.976213 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.976484 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.976525 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:38.977127 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.471928 (+494801us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.856927 (+384999us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.967424 (+110497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.012072 (+ 44648us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.863769 (+1851697us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.920142 (+ 56373us) raft_consensus.cc:1685] finished
0709 13:15:41.920272 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.920450 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8626,"prepare.queue_time_us":19900,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":36,"raft.queue_time_us":12322,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":32978,"spinlock_wait_cycles":380876544,"thread_start_us":808232,"threads_started":2,"wal-append.queue_time_us":790678}
W20260709 13:15:41.968086 13316 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:15:41.968712 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.949401 13301 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.936055 13287 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.954809 12883 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.962713 13080 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.983685 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1399) took 3401 ms. Trace:
I20260709 13:15:41.959470 13302 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:41.971530 13307 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.983656 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1400) took 3399 ms. Trace:
W20260709 13:15:41.984179 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56383.8us
W20260709 13:15:41.986537 13315 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.002s
I20260709 13:15:41.986794 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:42.001636 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.988389 11549 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:41.983604 13273 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.148s	user 0.010s	sys 0.006s
I20260709 13:15:41.992520 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:41.991161 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:41.989951 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 156) took 3125 ms. Trace:
I20260709 13:15:41.997567 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:41.990805 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 156) took 2833 ms. Trace:
I20260709 13:15:41.982215 13301 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:41.999176 11514 leader_election.cc:336] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.150s (SENT)
I20260709 13:15:41.999671 13305 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:41.976212 13178 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.978116 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:42.004351 11477 rpcz_store.cc:276] 0709 13:15:38.864606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.988885 (+3124279us) service_pool.cc:224] Handling call
0709 13:15:41.989293 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.989569 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.989599 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.989672 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:41.989716 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.989775 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.989868 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:41.999152 13276 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.407s	user 0.010s	sys 0.014s
I20260709 13:15:42.004597 11396 rpcz_store.cc:276] 0709 13:15:39.157314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.359584 (+202270us) service_pool.cc:224] Handling call
0709 13:15:41.901960 (+2542376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.919140 (+ 17180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.990600 (+ 71460us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b5400232150. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:41.990752 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5807187840}
I20260709 13:15:41.985219 11468 rpcz_store.cc:276] 0709 13:15:38.582642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.416516 (+833874us) service_pool.cc:224] Handling call
0709 13:15:39.416847 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.425652 (+  8805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.425701 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.426255 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.798578 (+372323us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.798635 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.798727 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.906413 (+2107686us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:41.981929 (+ 75516us) raft_consensus.cc:1685] finished
0709 13:15:41.981954 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.983508 (+  1554us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81226,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":14932,"spinlock_wait_cycles":296627968,"thread_start_us":353545,"threads_started":1,"wal-append.queue_time_us":717674}
W20260709 13:15:42.009670 11514 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.186s (SENT)
I20260709 13:15:42.010269 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 158) took 3128 ms. Trace:
I20260709 13:15:41.999267 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.004143 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 155) took 3128 ms. Trace:
I20260709 13:15:42.010648 11468 rpcz_store.cc:276] 0709 13:15:38.882092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.009300 (+3127208us) service_pool.cc:224] Handling call
0709 13:15:42.009697 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.009901 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.009924 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.009977 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:42.010006 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.010055 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.010190 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.011512 11543 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:41.996877 13302 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.010185 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:42.014607 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.001572 13307 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.010583 13178 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.997407 11473 rpcz_store.cc:276] 0709 13:15:38.583891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.568551 (+2984660us) service_pool.cc:224] Handling call
0709 13:15:41.634234 (+ 65683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.662947 (+ 28713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.662978 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.689466 (+ 26488us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.753666 (+ 64200us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:41.757070 (+  3404us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.757208 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.757284 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.862052 (+104768us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b1800218d40. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:15:41.980308 (+118256us) raft_consensus.cc:1685] finished
0709 13:15:41.983409 (+  3101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.983583 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1638,"prepare.queue_time_us":14807,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":7559,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":1938,"spinlock_wait_cycles":31639808,"thread_start_us":22657,"threads_started":1,"wal-append.queue_time_us":73664}
I20260709 13:15:42.025375 11469 raft_consensus.cc:3055] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:42.025740 13080 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.019888 13305 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:42.026273 11514 leader_election.cc:336] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.193s (SENT)
I20260709 13:15:42.007633 13301 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.026675 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 160) took 3045 ms. Trace:
I20260709 13:15:42.026772 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:42.027531 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:42.028219 11514 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.406s (SENT)
I20260709 13:15:42.028723 11475 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:42.031306 11545 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:42.023150 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.031911 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:42.032110 11471 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:42.032770 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:42.032801 13274 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:42.029879 13129 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:42.033416 13274 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.033504 13129 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.033860 13274 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.033952 13129 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.029939 13080 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.024313 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.025883 11469 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:42.024439 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1403) took 3199 ms. Trace:
I20260709 13:15:42.011181 11472 rpcz_store.cc:276] 0709 13:15:38.864350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.984068 (+3119718us) service_pool.cc:224] Handling call
0709 13:15:41.984419 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.984658 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.984699 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.984749 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:41.984777 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.984841 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.992564 (+  7723us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:42.035368 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.015507 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 157) took 3135 ms. Trace:
W20260709 13:15:42.029197 11514 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.471s (SENT)
I20260709 13:15:42.027132 11473 rpcz_store.cc:276] 0709 13:15:38.981758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.025824 (+3044066us) service_pool.cc:224] Handling call
0709 13:15:42.026178 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.026382 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.026411 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.026451 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:42.026484 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.026535 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.026628 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.029316 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 161) took 3008 ms. Trace:
I20260709 13:15:42.040853 11476 rpcz_store.cc:276] 0709 13:15:38.824873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.625933 (+2801060us) service_pool.cc:224] Handling call
0709 13:15:41.638849 (+ 12916us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.660627 (+ 21778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.660659 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.666687 (+  6028us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.754454 (+ 87767us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:41.795108 (+ 40654us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.795398 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.795467 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.024244 (+228777us) raft_consensus.cc:1685] finished
0709 13:15:42.024263 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.024385 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2497,"prepare.queue_time_us":40676,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":5281,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":1407,"thread_start_us":81830,"threads_started":2,"wal-append.queue_time_us":170311}
I20260709 13:15:42.045993 11477 rpcz_store.cc:276] 0709 13:15:38.879944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.008301 (+3128357us) service_pool.cc:224] Handling call
0709 13:15:42.010975 (+  2674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.011189 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.011224 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.011266 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:42.015230 (+  3964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.015314 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.015437 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.046789 11475 rpcz_store.cc:276] 0709 13:15:39.020916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.027859 (+3006943us) service_pool.cc:224] Handling call
0709 13:15:42.028350 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.029093 (+   743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.029267 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.053685 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:42.055567 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:42.056413 12112 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:42.057255 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.059429 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.060182 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.060530 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.061462 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.061774 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.062058 11551 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:42.042070 13199 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:42.063315 13199 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:42.063722 13199 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.068259 11475 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:42.072021 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1411) took 2862 ms. Trace:
I20260709 13:15:42.030565 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1404) took 3205 ms. Trace:
I20260709 13:15:42.072474 11475 rpcz_store.cc:276] 0709 13:15:39.209642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.067636 (+2857994us) service_pool.cc:224] Handling call
0709 13:15:42.067957 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.071536 (+  3579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.071960 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.072733 11470 rpcz_store.cc:276] 0709 13:15:38.825191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.680150 (+2854959us) service_pool.cc:224] Handling call
0709 13:15:41.680672 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.681388 (+   716us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:41.682197 (+   809us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.737654 (+ 55457us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.765343 (+ 27689us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.765502 (+   159us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:41.851844 (+ 86342us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.862796 (+ 10952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.862902 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.984400 (+121498us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:42.030343 (+ 45943us) raft_consensus.cc:1685] finished
0709 13:15:42.030365 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.030493 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3024,"prepare.queue_time_us":30071,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":9484,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":9986,"spinlock_wait_cycles":5407488,"thread_start_us":86096,"threads_started":2,"wal-append.queue_time_us":134242}
I20260709 13:15:42.074515 13305 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:42.076342 13160 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.065979 11474 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:42.077829 11477 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:42.064826 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.085952 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1416) took 2574 ms. Trace:
I20260709 13:15:42.086557 11474 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:42.086527 11472 rpcz_store.cc:276] 0709 13:15:39.511685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.084942 (+2573257us) service_pool.cc:224] Handling call
0709 13:15:42.085343 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.085604 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.085641 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.085689 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:42.085722 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.085784 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.085885 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.081499 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1415) took 2641 ms. Trace:
I20260709 13:15:42.087911 11477 rpcz_store.cc:276] 0709 13:15:39.439975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.077141 (+2637166us) service_pool.cc:224] Handling call
0709 13:15:42.077461 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.081255 (+  3794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.081447 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.008139 13286 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.087505 13052 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.095211 11398 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.087121 11474 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:42.098131 13302 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.047425 13188 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.101552 12923 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:15:42.110478 12855 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:42.115741 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86573.2us
I20260709 13:15:42.077687 13301 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.108565 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:42.120332 11397 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:42.115890 11469 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.128933 11395 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:42.122234 11514 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.646s (SENT)
I20260709 13:15:42.109270 13113 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.146766 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1417) took 2084 ms. Trace:
I20260709 13:15:42.147339 11477 rpcz_store.cc:276] 0709 13:15:40.062421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.120850 (+2058429us) service_pool.cc:224] Handling call
0709 13:15:42.131359 (+ 10509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.143982 (+ 12623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.144013 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.146419 (+  2406us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:42.146475 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.146556 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.146696 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.123610 13307 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.131698 13203 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:42.163986 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44772.4us
I20260709 13:15:42.160395 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.146142 13287 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.165601 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:42.171406 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43724.5us
I20260709 13:15:42.182003 11399 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.183245 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.140409 13305 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:15:42.192314 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.195292 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.198525 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:42.198680 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1413) took 2806 ms. Trace:
I20260709 13:15:42.199386 11475 rpcz_store.cc:276] 0709 13:15:39.389155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.074562 (+2685407us) service_pool.cc:224] Handling call
0709 13:15:42.075533 (+   971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.075800 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:42.077320 (+  1520us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.195620 (+118300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.195668 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:42.195694 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.195738 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.195815 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":990,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"thread_start_us":109901,"threads_started":1}
I20260709 13:15:42.189167 11470 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:42.196877 11402 raft_consensus.cc:2376] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3. Re-sending same reply.
I20260709 13:15:42.208734 11469 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:42.215708 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.216439 12115 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:42.218899 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.219424 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:42.218763 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.220140 11543 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:42.221108 11396 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4. Re-sending same reply.
I20260709 13:15:42.222723 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.209110 13286 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.224277 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.209761 11470 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:42.210892 12855 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:42.223308 11398 raft_consensus.cc:2376] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4. Re-sending same reply.
I20260709 13:15:42.225464 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.209399 13160 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.226193 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:42.226466 11402 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.228967 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.229812 13307 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.230243 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.230406 11470 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.237432 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:42.231086 13322 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:15:42.238694 11470 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.246444 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.222914 13302 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.272785 11396 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:42.227522 12855 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.214124 13173 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.227913 13188 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.239378 13301 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.267475 13286 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:15:42.317473 11395 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:42.319962 12123 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:42.224563 13178 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.301172 13265 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:42.332737 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:42.336251 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.276292 12855 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.349490 11401 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:42.347563 13331 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:15:42.361383 11398 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.360441 11469 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:42.362119 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 159) took 3464 ms. Trace:
I20260709 13:15:42.362452 11469 rpcz_store.cc:276] 0709 13:15:38.897333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.019392 (+3122059us) service_pool.cc:224] Handling call
0709 13:15:42.019850 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.361891 (+342041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.362069 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.364205 11475 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:42.274042 13030 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:42.365944 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 180527us
I20260709 13:15:42.283993 13274 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.374017 13200 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:42.374567 13200 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:42.374924 13300 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.520s	user 0.014s	sys 0.000s
I20260709 13:15:42.375368 13300 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.375566 13200 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.377430 11400 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:42.370779 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:42.378924 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.370414 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.360896 13265 consensus_queue.cc:1243] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:42.358053 13113 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.385900 11477 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:42.388245 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.378767 13330 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
W20260709 13:15:42.427407 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.432861 11545 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:15:42.437263 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.444474 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.371519 13329 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:15:42.447085 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.448915 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:42.464368 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.477722 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.470944 11394 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.485370 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.486123 11543 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:42.486424 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.488000 11545 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:42.490233 13338 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:15:42.490464 13300 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:42.505437 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.512657 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1407) took 3388 ms. Trace:
W20260709 13:15:42.516672 13341 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:15:42.511920 13300 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:15:42.524996 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1412) took 3151 ms. Trace:
I20260709 13:15:42.525461 11473 rpcz_store.cc:276] 0709 13:15:39.373406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.069252 (+2695846us) service_pool.cc:224] Handling call
0709 13:15:42.069615 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:42.070130 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.070157 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:42.070687 (+   530us) log.cc:844] Serialized 52 byte log entry
0709 13:15:42.124074 (+ 53387us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:42.124125 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.124281 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.524745 (+400464us) raft_consensus.cc:1685] finished
0709 13:15:42.524766 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.524931 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1223,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":53011,"threads_started":1,"wal-append.queue_time_us":116151}
I20260709 13:15:42.528693 13203 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:42.529167 13203 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:42.530006 13203 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.533151 11474 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.534502 11474 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.513033 11468 rpcz_store.cc:276] 0709 13:15:39.124805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.048384 (+2923579us) service_pool.cc:224] Handling call
0709 13:15:42.048772 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.049152 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.049179 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:42.049781 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 13:15:42.191170 (+141389us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:42.210299 (+ 19129us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.512336 (+302037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.512416 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.512457 (+    41us) raft_consensus.cc:1685] finished
0709 13:15:42.512471 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.512600 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288,"prepare.queue_time_us":18929,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":14893,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":3850,"thread_start_us":442152,"threads_started":2,"wal-append.queue_time_us":196033}
W20260709 13:15:42.541247 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 217988us
I20260709 13:15:42.541798 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:42.542888 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:42.547765 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.550300 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:42.559227 13345 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:15:42.540310 13173 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:42.562211 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.489050 13198 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:15:42.512532 13129 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.563390 13203 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.547459 13307 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:42.499336 13199 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:42.572863 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.573632 13174 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.577607 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.558840 13334 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:42.554679 13286 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:42.544075 13305 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:15:42.581907 13350 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:15:42.582540 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.583128 13350 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:42.583395 11543 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:42.583535 13350 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:42.584921 13199 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:42.567830 13178 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.588075 11546 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:42.589435 13326 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 13:15:42.597366 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:42.599920 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.594619 11397 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.597541 11477 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:42.600720 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.601191 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.601537 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.602062 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.592865 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.576615 13307 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.577078 13274 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:42.608505 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.595479 13286 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.610366 13304 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:42.610813 13304 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:42.611737 13304 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.612771 13304 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:42.589612 13327 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.505s	user 0.002s	sys 0.000s
I20260709 13:15:42.615386 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:42.616147 13030 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.606936 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.616348 11398 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:42.621325 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.622781 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:42.625260 11401 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:15:42.605942 13307 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.628958 11470 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:15:42.629382 11470 pending_rounds.cc:85] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 1
I20260709 13:15:42.629626 11470 pending_rounds.cc:107] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:42.629863 11470 raft_consensus.cc:2889] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:42.630903 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 925) took 2797 ms. Trace:
I20260709 13:15:42.631300 11470 rpcz_store.cc:276] 0709 13:15:39.833168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.107331 (+2274163us) service_pool.cc:224] Handling call
0709 13:15:42.161572 (+ 54241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.630638 (+469066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.630827 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:42.632608 13129 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:42.625209 12855 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:42.635155 13343 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:15:42.635771 12112 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:42.636453 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1408) took 3511 ms. Trace:
I20260709 13:15:42.636853 11476 rpcz_store.cc:276] 0709 13:15:39.125174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.048626 (+2923452us) service_pool.cc:224] Handling call
0709 13:15:42.049006 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:42.083497 (+ 34491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.083525 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:42.084128 (+   603us) log.cc:844] Serialized 52 byte log entry
0709 13:15:42.345999 (+261871us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:42.346053 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.346149 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.636225 (+290076us) raft_consensus.cc:1685] finished
0709 13:15:42.636250 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.636394 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13483,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":269232,"threads_started":2,"wal-append.queue_time_us":297997}
W20260709 13:15:42.634793 13325 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.552s	user 0.002s	sys 0.001s
I20260709 13:15:42.637516 12115 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
W20260709 13:15:42.641981 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.642870 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.644006 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0824694620000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 13:15:42.610857 13286 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:42.646713 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.637449 12855 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:42.650776 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.651554 11549 raft_consensus.cc:2368] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6.
I20260709 13:15:42.654040 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:42.654811 11543 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:42.656330 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.632642 11470 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:15:42.644892 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0392694619999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:15:42.660455 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:42.675576 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:42.687521 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0320694619999813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 13:15:42.688177 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0248694619999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 13:15:42.688714 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0212694619999638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:15:42.689163 11442 connection.cc:256] RPC call timeout handler was delayed by -3.0212694619999638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:15:42.689596 11442 connection.cc:256] RPC call timeout handler was delayed by -2.968169461999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:15:42.690018 11442 connection.cc:256] RPC call timeout handler was delayed by -2.9303694620000442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:15:42.690423 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8763694619999569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
W20260709 13:15:42.691021 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8763694619999569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
I20260709 13:15:42.693571 11474 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:42.694082 11474 pending_rounds.cc:85] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 1
I20260709 13:15:42.694375 11474 pending_rounds.cc:107] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:42.694684 11474 raft_consensus.cc:2889] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:42.695520 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:42.691615 11442 connection.cc:256] RPC call timeout handler was delayed by -2.854769461999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 13:15:42.589201 13200 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:15:42.705169 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:42.705901 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.706583 11470 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.708266 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.708891 12118 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.702625 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.711272 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.711818 12113 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.715095 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.715893 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:42.710397 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8511694619999162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:15:42.718506 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8439694619999045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:15:42.721163 12119 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:42.722290 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.726413 12122 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:42.729849 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8439694619999045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:15:42.730935 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8133694620000824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 13:15:42.734822 11442 connection.cc:256] RPC call timeout handler was delayed by -2.8133694620000824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
W20260709 13:15:42.736064 11442 connection.cc:256] RPC call timeout handler was delayed by -2.792669462000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:15:42.738883 11442 connection.cc:256] RPC call timeout handler was delayed by -2.792669462000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
I20260709 13:15:42.738161 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:42.744472 12115 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.749851 11551 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:42.738626 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 920) took 2976 ms. Trace:
I20260709 13:15:42.753937 11474 rpcz_store.cc:276] 0709 13:15:39.761961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.086029 (+2324068us) service_pool.cc:224] Handling call
0709 13:15:42.086386 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.532629 (+446243us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b4c0040e4c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da7be5a20 00007f8da7bf6a29 00007f8da7bbc644 00007f8da7be3f2d 00007f8da7be3b93 00007f8da991418b 00007f8da9914cb5 00007f8da990ff30 00007f8da99117de 00007f8da991bb49 00007f8da991c416
0709 13:15:42.723950 (+191321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.737030 (+ 13080us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11043584}
W20260709 13:15:42.763381 11442 connection.cc:256] RPC call timeout handler was delayed by -2.7431694619999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:15:42.763959 11442 connection.cc:256] RPC call timeout handler was delayed by -2.7431694619999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:15:42.764391 11442 connection.cc:256] RPC call timeout handler was delayed by -2.7359694619999573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 13:15:42.764834 11442 connection.cc:256] RPC call timeout handler was delayed by -2.7359694619999573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 13:15:42.765244 11442 connection.cc:256] RPC call timeout handler was delayed by -2.711669461999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
W20260709 13:15:42.766510 11442 connection.cc:256] RPC call timeout handler was delayed by -2.7044694619999063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 13:15:42.767410 11442 connection.cc:256] RPC call timeout handler was delayed by -2.6864694619998772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 13:15:42.768324 11442 connection.cc:256] RPC call timeout handler was delayed by -2.6324694620000173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:15:42.774714 11442 connection.cc:256] RPC call timeout handler was delayed by -2.6306694620000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 13:15:42.779588 11442 connection.cc:256] RPC call timeout handler was delayed by -2.6171694619999926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 13:15:42.780076 11442 connection.cc:256] RPC call timeout handler was delayed by -2.5199694620000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
W20260709 13:15:42.789758 11442 connection.cc:256] RPC call timeout handler was delayed by -2.5199694620000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
W20260709 13:15:42.790702 11442 connection.cc:256] RPC call timeout handler was delayed by -2.516369462000057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 13:15:42.791152 11442 connection.cc:256] RPC call timeout handler was delayed by -2.5037694620000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 13:15:42.791571 11442 connection.cc:256] RPC call timeout handler was delayed by -2.5001694620000308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 13:15:42.795113 11442 connection.cc:256] RPC call timeout handler was delayed by -2.5001694620000308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 13:15:42.796345 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4965694620000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:15:42.799948 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4884694620000118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
W20260709 13:15:42.800560 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4569694619999609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:15:42.806133 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4569694619999609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:15:42.806900 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4470694619999449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:15:42.807439 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4470694619999449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:15:42.807965 11442 connection.cc:256] RPC call timeout handler was delayed by -2.2904694619999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2389s for a response
W20260709 13:15:42.812752 11442 connection.cc:256] RPC call timeout handler was delayed by -2.2904694619999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2389s for a response
W20260709 13:15:42.813331 11442 connection.cc:256] RPC call timeout handler was delayed by -2.2373694620000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24480000000000002s for a response
W20260709 13:15:42.813848 11442 connection.cc:256] RPC call timeout handler was delayed by -2.2373694620000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24480000000000002s for a response
I20260709 13:15:42.824364 11545 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:42.825722 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:42.826395 12118 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:42.833976 11400 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:42.837109 12115 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:15:42.845928 13328 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.762s	user 0.004s	sys 0.000s
I20260709 13:15:42.850370 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1405) took 3760 ms. Trace:
I20260709 13:15:42.850919 11471 rpcz_store.cc:276] 0709 13:15:39.090008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.032858 (+2942850us) service_pool.cc:224] Handling call
0709 13:15:42.033393 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.033899 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.033928 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:42.034449 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:15:42.083795 (+ 49346us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:42.083835 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.083906 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.845302 (+761396us) spinlock_profiling.cc:243] Waited 8.67 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:42.849921 (+  4619us) raft_consensus.cc:1685] finished
0709 13:15:42.849953 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.850277 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1082,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":19947904,"thread_start_us":48885,"threads_started":1,"wal-append.queue_time_us":49287}
I20260709 13:15:42.842427 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:42.864280 13307 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.876794 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:42.877859 13353 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:42.878034 13350 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.880374 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:42.879266 13350 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.884812 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.888236 11551 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.888432 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.888980 11545 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:42.891116 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.894043 13076 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.886042 13307 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:42.881381 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:42.899744 11551 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:42.901310 11545 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:42.905495 13203 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:42.887181 13353 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:42.919148 13258 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:42.908461 13203 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:42.910727 13355 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:15:42.938939 12923 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:42.939561 12923 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:42.940656 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:42.941967 13274 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:42.943214 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:42.943243 13349 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:42.940688 12923 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.946790 12923 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.943598 13335 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:42.947515 13335 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:42.948380 13335 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:42.949640 13203 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:42.951172 13203 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.906050 13129 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.962044 13129 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:42.963225 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 81969.6us
I20260709 13:15:42.924221 13353 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:42.905200 13310 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:15:42.976492 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:43.010545 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:43.012019 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:43.000880 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.977392 11442 leader_election.cc:400] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 4.
I20260709 13:15:43.023670 11442 leader_election.cc:403] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:15:42.999399 11514 leader_election.cc:336] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.134s (SENT)
W20260709 13:15:43.036430 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59251us
I20260709 13:15:43.039520 12085 leader_election.cc:400] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6.
I20260709 13:15:43.040016 12085 leader_election.cc:403] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:15:43.042428 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.922639 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:42.921962 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:43.049109 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.049786 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 127899us
I20260709 13:15:42.972208 11394 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:42.976989 12923 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:43.036042 13350 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:43.049160 11545 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:43.056908 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57554us
I20260709 13:15:43.056193 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.054486 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:42.965461 13129 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:43.034868 13274 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:42.996737 13286 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.976025 13353 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.985409 13265 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.061443 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:43.040118 13310 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:42.968909 13200 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:15:43.041417 13258 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 4.
I20260709 13:15:43.066433 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:42.976713 13349 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:15:43.068073 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.055859 13307 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:43.055703 12923 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:43.060408 12804 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6.
I20260709 13:15:43.079114 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.074724 13113 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:43.073189 13307 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:43.063221 13348 raft_consensus.cc:2764] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:15:43.077765 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.077975 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:43.078493 13178 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:43.066462 13329 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:43.070279 12855 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:43.087170 13250 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.614s	user 0.006s	sys 0.008s
I20260709 13:15:43.083854 13113 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:43.065279 13286 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.086733 13307 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.087858 13305 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:43.087667 13351 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:43.068161 13265 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:43.091130 13362 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:15:43.072562 13335 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:15:43.089633 13310 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:43.100796 11514 leader_election.cc:336] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.018s (SENT)
I20260709 13:15:43.103330 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:43.087774 13207 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:15:43.107160 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.107324 11545 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:43.090245 12855 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:43.122529 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:43.107892 13351 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:43.128078 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:43.136593 13160 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:43.139117 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.139751 11473 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.117009 12923 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.142040 12923 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.143299 13113 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.144513 13113 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.145983 13353 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:43.148138 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.149552 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.150269 11471 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:43.120342 12855 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.153123 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.153723 11470 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:43.156510 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:43.112037 13336 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:15:43.157876 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.158845 13178 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:43.159598 13178 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:43.160782 13178 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.148142 13286 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:43.165127 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:43.166599 11470 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:43.167169 13258 raft_consensus.cc:2764] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:15:43.121002 13310 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:43.122341 13329 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:43.173215 13265 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:43.173980 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:43.130384 13351 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:43.179224 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.183027 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.213346 11469 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:43.184417 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.185581 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:43.188594 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:43.188894 13353 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:43.194191 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.199316 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 98576.6us
I20260709 13:15:43.194671 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.204221 11473 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:43.226297 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:43.221778 11514 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.180s (SENT)
I20260709 13:15:43.209561 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:43.217576 11470 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:43.219017 11475 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:43.203799 13076 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:43.207782 13329 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:43.241453 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.254251 12118 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:43.261235 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 51777us
W20260709 13:15:43.262979 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.264091 11469 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:43.268373 12117 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:43.269311 13076 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:43.269783 13076 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:43.270604 13076 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.271611 13030 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:43.271636 12122 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:43.274758 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:43.275475 11546 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:43.277441 12113 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:43.278728 11299 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 2 to 3, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.220198 11393 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:43.293092 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:43.255914 13329 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.228518 13159 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:43.241756 13258 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:43.239147 13335 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:43.298796 11514 leader_election.cc:336] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.545s (SENT)
I20260709 13:15:43.309361 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:43.281306 11299 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.285692 11471 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
W20260709 13:15:43.312294 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.328181 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.302973 13258 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:43.333115 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:43.235731 13310 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.295543 13030 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:15:43.323472 13203 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:43.371448 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 76711.7us
W20260709 13:15:43.376338 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 209369us
I20260709 13:15:43.377215 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.378139 13353 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:43.378618 13353 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:43.379056 13353 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.381861 12855 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.335319 13334 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:15:43.308032 13307 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.389791 13307 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:43.393286 11514 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.802s (SENT)
I20260709 13:15:43.396124 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.398938 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:43.400002 11297 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 140) took 12863 ms (client timeout 15000 ms). Trace:
I20260709 13:15:43.401091 13353 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.402498 11299 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.403818 13353 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:43.404428 13353 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:43.405706 13353 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:43.406138 13353 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:43.408771 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:43.409349 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 277902us
I20260709 13:15:43.410030 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:43.365757 13258 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.413321 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:43.373942 13336 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:43.431631 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:43.493937 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 121840us
W20260709 13:15:43.494009 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62465.2us
W20260709 13:15:43.495628 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64805.9us
W20260709 13:15:43.502663 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.502631 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:43.503726 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.514918 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 181917us
I20260709 13:15:43.516366 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:43.420384 13203 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:43.517596 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 120083us
I20260709 13:15:43.422458 13159 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:43.522955 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:43.521992 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:43.422852 13030 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 13:15:43.516112 13080 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:43.535769 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:43.524173 13305 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:43.525312 13348 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:43.552201 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.541326 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.546779 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:43.548935 13113 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:43.553231 11514 leader_election.cc:336] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.116s (SENT)
I20260709 13:15:43.425300 13353 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:43.536890 13336 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:43.400480 11297 rpcz_store.cc:269] 0709 13:15:30.536098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.542105 (+  6007us) service_pool.cc:224] Handling call
0709 13:15:43.399779 (+12857674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.320851 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.378235 (+ 57384us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.379750 (+  1515us) write_op.cc:435] Acquired schema lock
0709 13:15:39.381516 (+  1766us) tablet.cc:662] Decoding operations
0709 13:15:39.496359 (+114843us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.496445 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.496482 (+    37us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:15:39.497201 (+   719us) tablet.cc:701] Row locks acquired
0709 13:15:39.497222 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:15:39.497374 (+   152us) write_op.cc:270] Start()
0709 13:15:39.508501 (+ 11127us) write_op.cc:276] Timestamp: P: 1783602939497329 usec, L: 0
0709 13:15:39.518505 (+ 10004us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.647420 (+128915us) log.cc:844] Serialized 2772 byte log entry
0709 13:15:41.614627 (+1967207us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da680f0ac 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b
0709 13:15:42.106138 (+491511us) op_driver.cc:464] REPLICATION: finished
0709 13:15:42.253868 (+147730us) write_op.cc:301] APPLY: starting
0709 13:15:42.254048 (+   180us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:42.254426 (+   378us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:42.254500 (+    74us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:42.255792 (+  1292us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:42.256222 (+   430us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:15:42.256310 (+    88us) write_op.cc:312] APPLY: finished
0709 13:15:42.256851 (+   541us) log.cc:844] Serialized 59 byte log entry
0709 13:15:42.257076 (+   225us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:42.257313 (+   237us) write_op.cc:454] Released schema lock
0709 13:15:42.257394 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:42.259911 (+  2517us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:42.259953 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":133923,"apply.run_cpu_time_us":4104,"apply.run_wall_time_us":6054,"num_ops":7,"prepare.queue_time_us":305444,"prepare.run_cpu_time_us":64537,"prepare.run_wall_time_us":2442001,"raft.queue_time_us":126003,"raft.run_cpu_time_us":12700,"raft.run_wall_time_us":204605,"replication_time_us":2563058,"spinlock_wait_cycles":800143616,"thread_start_us":2298960,"threads_started":4,"wal-append.queue_time_us":2053756,"wal-append.run_cpu_time_us":38519,"wal-append.run_wall_time_us":1555664}]]}
I20260709 13:15:43.562737 11394 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:43.543604 12923 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 13:15:43.563468 13383 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:15:43.564182 13307 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:43.564608 13305 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:43.555891 12855 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:43.570345 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:43.537621 13080 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.576707 13055 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:43.574190 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.594761 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:43.607980 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:43.557973 13113 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:43.560563 13348 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:43.637923 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 120984us
W20260709 13:15:43.651419 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 58838us
I20260709 13:15:43.593624 13307 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:43.568677 13203 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:15:43.593683 13080 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.609555 13373 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:43.639503 13113 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:43.711513 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 158312us
W20260709 13:15:43.889554 11514 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.260s (SENT)
I20260709 13:15:43.892035 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:43.642544 13336 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:43.898180 13336 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.899945 11548 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:43.899535 13307 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:43.691282 13369 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:15:43.801460 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.902920 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.903403 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.786175 13348 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.825564 13373 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:43.745111 13305 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.684254 13349 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:15:43.919687 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 489654us
I20260709 13:15:43.884794 13258 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:43.690791 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:43.912636 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 317955us
W20260709 13:15:43.898738 13329 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.001s
I20260709 13:15:43.978354 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:43.939160 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:43.919832 13351 consensus_meta.cc:287] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.691s	user 0.007s	sys 0.006s
I20260709 13:15:43.921379 13349 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:43.927007 13380 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.393s	user 0.005s	sys 0.000s
W20260709 13:15:43.926262 13395 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.279s	user 0.001s	sys 0.004s
I20260709 13:15:43.921584 13373 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:43.920463 13394 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.228s	user 0.003s	sys 0.000s
W20260709 13:15:43.981145 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:43.985723 11474 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:43.990830 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51790.6us
I20260709 13:15:43.952558 13353 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:43.987998 13402 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:43.978657 13329 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:43.990142 13411 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.028558 13411 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.030865 13207 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:44.031471 13207 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.031962 13207 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:44.033208 13403 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:44.033649 13403 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.033984 13403 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:44.035815 13349 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:44.011998 13373 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.041261 13373 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:43.990388 13351 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:15:44.027700 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49437.8us
I20260709 13:15:44.043614 13351 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:44.044173 13351 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.011307 12923 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:44.026129 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92749.6us
I20260709 13:15:44.047923 11470 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:15:44.048194 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:44.030726 13411 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:44.044626 13351 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:43.986964 13080 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.065708 11476 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:44.057186 13402 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:44.064220 12923 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:44.096944 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:44.099367 11549 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:44.085140 12883 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:44.071893 13381 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:44.103554 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.104867 13381 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:44.121832 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 452037us
I20260709 13:15:44.102799 13203 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.130787 11551 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:44.110630 13055 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.116372 12883 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:44.122486 13385 raft_consensus.cc:476] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 13:15:44.141628 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:44.127038 13381 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:44.135259 13415 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.159246 13415 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.130163 13411 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.174278 13258 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:44.153319 11546 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:44.154907 13418 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.131381 13203 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.153440 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:44.185333 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.137432 13055 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:44.153311 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.178565 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:44.197005 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.013s
W20260709 13:15:44.197520 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.000s	sys 0.014s
W20260709 13:15:44.203491 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 193571us
W20260709 13:15:44.232930 11514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.099s (SENT) [suppressed 15 similar messages]
W20260709 13:15:44.240785 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 116857us
I20260709 13:15:44.232867 13160 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:44.146531 12923 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:44.245096 13258 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:44.185390 13411 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.282737 12118 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:44.124372 13402 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:44.300438 11543 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:44.350409 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:44.255496 13329 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:44.377441 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 144518us
W20260709 13:15:44.379508 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.296391 13080 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:44.400527 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.305397 13258 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.204141 13418 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:44.432204 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 81913.4us
I20260709 13:15:44.409453 13265 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.408607 13426 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.426277 13258 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:44.279641 12883 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.420948 13385 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:44.515221 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 165436us
W20260709 13:15:44.516702 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 274897us
I20260709 13:15:44.431607 13413 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.421037 13373 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:44.493235 13329 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:44.516880 12113 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:15:44.535228 11514 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.525s (SENT)
I20260709 13:15:44.550570 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:44.493422 13415 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:44.617578 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.653790 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 134776us
I20260709 13:15:44.657613 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 185) took 1016 ms. Trace:
W20260709 13:15:44.658062 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 122858us
W20260709 13:15:44.539951 13113 consensus_meta.cc:287] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.001s
I20260709 13:15:44.517670 13265 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:44.665655 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.555138 13353 consensus_meta.cc:287] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.002s
I20260709 13:15:44.675951 13265 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:44.683928 13428 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:15:44.684439 13388 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:44.685042 13388 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:44.520128 13403 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.526854 13373 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.661877 13410 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:44.667483 13113 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.712628 11442 leader_election.cc:400] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:44.713066 11442 leader_election.cc:403] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:44.534719 13438 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.180s
I20260709 13:15:44.711848 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:44.534550 13385 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.517804 13426 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.694075 13030 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:44.554813 13421 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.172s
I20260709 13:15:44.523523 13413 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:44.738612 13424 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.347s	user 0.001s	sys 0.001s
I20260709 13:15:44.738512 13329 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:44.761049 13203 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:44.756824 11477 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:44.777370 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 184) took 1182 ms. Trace:
I20260709 13:15:44.777791 11469 rpcz_store.cc:276] 0709 13:15:43.594839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.594945 (+   106us) service_pool.cc:224] Handling call
0709 13:15:43.632251 (+ 37306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.632472 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.632497 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.644926 (+ 12429us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.237926 (+593000us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:44.237969 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.238063 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.777148 (+539085us) raft_consensus.cc:1685] finished
0709 13:15:44.777167 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.777289 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":411706,"threads_started":1,"wal-append.queue_time_us":602683}
I20260709 13:15:44.755889 13265 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.783439 13265 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.760588 13030 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:44.789645 13030 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:44.753822 13435 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 13:15:44.686087 13388 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:44.658215 11472 rpcz_store.cc:276] 0709 13:15:43.641664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.643737 (+  2073us) service_pool.cc:224] Handling call
0709 13:15:43.886159 (+242422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.886467 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:44.324661 (+438194us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.548914 (+224253us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.554337 (+  5423us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:44.657218 (+102881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.657334 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.657527 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119358,"prepare.queue_time_us":594683,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":132051,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":22134,"thread_start_us":440050,"threads_started":2}
I20260709 13:15:44.802177 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:44.806950 13439 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:15:44.809501 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 108999us
I20260709 13:15:44.810549 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:44.795099 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:44.820142 13422 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
I20260709 13:15:44.802861 11472 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:44.821117 12117 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:44.831312 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:44.831924 13417 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.661s	user 0.007s	sys 0.000s
I20260709 13:15:44.842499 11472 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:44.844575 11472 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:44.826917 13388 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:44.820290 13450 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:44.842582 13379 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.954s	user 0.002s	sys 0.000s
I20260709 13:15:44.830072 12113 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:44.857127 13030 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:15:44.858647 13438 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 13:15:44.871398 13436 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:15:44.873113 13113 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:44.875087 13405 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
W20260709 13:15:44.879808 13389 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.940s	user 0.000s	sys 0.002s
I20260709 13:15:44.879591 13451 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:44.885349 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.970549 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.887946 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:44.849956 13445 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:44.857934 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:44.987694 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 134443us
W20260709 13:15:44.998553 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.972846 13452 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.977928 13445 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:45.003682 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.014699 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 159) took 1741 ms. Trace:
I20260709 13:15:44.991787 13412 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:44.992916 13113 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:44.984706 13265 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.012872 13392 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:45.041015 13450 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.011845 13159 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.039417 13444 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:45.100250 13444 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:45.030057 13386 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.142s	user 0.000s	sys 0.008s
I20260709 13:15:45.062741 13445 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:45.043238 13385 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.056957 11547 rpcz_store.cc:276] 0709 13:15:43.273435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.276106 (+  2671us) service_pool.cc:224] Handling call
0709 13:15:43.276442 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.431621 (+155179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.431664 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.516431 (+ 84767us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.687558 (+171127us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:43.687601 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.687682 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.014472 (+1326790us) raft_consensus.cc:1685] finished
0709 13:15:45.014500 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.014641 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1873,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":73,"thread_start_us":290613,"threads_started":2,"wal-append.queue_time_us":714547}
W20260709 13:15:45.095464 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.038568 13378 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:15:45.099236 13447 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:15:45.021338 13055 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.079393 13200 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.064651 13412 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.064635 13403 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:45.129014 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 131924us
I20260709 13:15:45.102293 13329 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:45.076894 13373 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.130920 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:44.982435 13451 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:45.102550 13307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.017s	sys 0.011s
I20260709 13:15:45.101305 13444 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.126221 13378 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.163745 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:45.177184 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.133347 13426 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:45.144804 13307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.238s	user 0.017s	sys 0.011s
I20260709 13:15:45.197466 13444 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:45.205149 13349 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:45.209506 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1468) took 1071 ms. Trace:
I20260709 13:15:45.210852 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 182) took 1654 ms. Trace:
I20260709 13:15:45.104185 13392 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.211369 11474 rpcz_store.cc:276] 0709 13:15:43.556702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.557283 (+   581us) service_pool.cc:224] Handling call
0709 13:15:43.562776 (+  5493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.015043 (+452267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.210554 (+1195511us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400243e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:45.210765 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972005760}
I20260709 13:15:45.213027 13392 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.213405 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1471) took 1152 ms. Trace:
I20260709 13:15:45.213822 11474 rpcz_store.cc:276] 0709 13:15:44.061318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.212484 (+1151166us) service_pool.cc:224] Handling call
0709 13:15:45.212817 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.213023 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.213056 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.213105 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:45.213148 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.213211 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.213326 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:45.215437 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 185) took 1995 ms. Trace:
I20260709 13:15:45.215828 11395 rpcz_store.cc:276] 0709 13:15:43.219674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.219889 (+   215us) service_pool.cc:224] Handling call
0709 13:15:43.220220 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:43.245490 (+ 25270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.245518 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:43.288088 (+ 42570us) log.cc:844] Serialized 52 byte log entry
0709 13:15:43.367189 (+ 79101us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:43.376573 (+  9384us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.554632 (+178059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.554725 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.208903 (+1654178us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b2400137078. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:15:45.215135 (+  6232us) raft_consensus.cc:1685] finished
0709 13:15:45.215221 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.215355 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49529,"prepare.queue_time_us":8916,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":67674,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":100189,"spinlock_wait_cycles":5298560,"thread_start_us":197396,"threads_started":2,"wal-append.queue_time_us":222703}
W20260709 13:15:45.219441 13419 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.048s	user 0.003s	sys 0.000s
I20260709 13:15:45.221043 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 183) took 1628 ms. Trace:
I20260709 13:15:45.221416 11471 rpcz_store.cc:276] 0709 13:15:43.593108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.609877 (+ 16769us) service_pool.cc:224] Handling call
0709 13:15:43.669835 (+ 59958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.670168 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.670211 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.670824 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.112393 (+441569us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:44.393775 (+281382us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.668227 (+274452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.668315 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.030701 (+362386us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:45.220723 (+190022us) raft_consensus.cc:1685] finished
0709 13:15:45.220831 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.220982 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37705,"prepare.queue_time_us":578619,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":53,"raft.queue_time_us":88496,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":33443,"spinlock_wait_cycles":8886656,"thread_start_us":609956,"threads_started":3,"wal-append.queue_time_us":454965}
I20260709 13:15:45.223155 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1454) took 2040 ms. Trace:
I20260709 13:15:45.223567 11476 rpcz_store.cc:276] 0709 13:15:43.182302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.183562 (+  1260us) service_pool.cc:224] Handling call
0709 13:15:43.183878 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.078326 (+894448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.222903 (+1144577us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540039b7d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:45.223093 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027433344}
I20260709 13:15:45.224154 13411 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:45.224737 13381 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.048s	user 0.006s	sys 0.006s
I20260709 13:15:45.226183 13412 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:45.228762 13443 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.423s	user 0.000s	sys 0.001s
W20260709 13:15:45.259521 13207 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.227s	user 0.012s	sys 0.002s
I20260709 13:15:45.259949 13207 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.210080 11546 rpcz_store.cc:276] 0709 13:15:44.138253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.193488 (+ 55235us) service_pool.cc:224] Handling call
0709 13:15:44.193757 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.489941 (+296184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.489973 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.490020 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:44.490059 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.209176 (+719117us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540026a054. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:45.209233 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.209419 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680663296}
I20260709 13:15:45.144661 13055 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:45.164672 13113 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.232303 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 179) took 2037 ms. Trace:
I20260709 13:15:45.236816 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1463) took 1185 ms. Trace:
W20260709 13:15:45.263335 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:45.263273 11477 rpcz_store.cc:276] 0709 13:15:44.051507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.690114 (+638607us) service_pool.cc:224] Handling call
0709 13:15:44.690503 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.777520 (+ 87017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.236538 (+459018us) spinlock_profiling.cc:243] Waited 66 ms on lock 0x7b54003895d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:45.236742 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151886592}
I20260709 13:15:45.064795 13452 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.237689 11476 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:45.124879 13159 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:45.255780 13453 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:45.257650 13411 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.276458 13411 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.277262 13351 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.277626 13452 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.133939 13385 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:15:45.229117 13174 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:45.261747 11468 rpcz_store.cc:276] 0709 13:15:43.195214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.203617 (+  8403us) service_pool.cc:224] Handling call
0709 13:15:43.216048 (+ 12431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.243302 (+ 27254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.243344 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.257622 (+ 14278us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.416958 (+159336us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:43.417012 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.417113 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.029342 (+1612229us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:45.231942 (+202600us) raft_consensus.cc:1685] finished
0709 13:15:45.231974 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.232246 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11612,"raft.queue_time_us":26896,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":19098,"spinlock_wait_cycles":131052032,"thread_start_us":114731,"threads_started":1,"wal-append.queue_time_us":396818}
W20260709 13:15:45.265708 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.268911 11476 pending_rounds.cc:85] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 1
I20260709 13:15:45.139429 13200 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.190081 13423 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.273629 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.279228 11476 pending_rounds.cc:107] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 13:15:45.279511 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:45.254844 13258 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.000s
I20260709 13:15:45.275688 13159 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.124476 13450 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.309514 13450 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.311527 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1460) took 1265 ms. Trace:
I20260709 13:15:45.311908 11473 rpcz_store.cc:276] 0709 13:15:44.046497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.046607 (+   110us) service_pool.cc:224] Handling call
0709 13:15:44.046856 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.047072 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:44.324795 (+277723us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.228137 (+903342us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:45.234762 (+  6625us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b180004a980. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8da9911060 00007f8da9923716 00007f8da9934287
0709 13:15:45.273441 (+ 38679us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.311178 (+ 37737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.311229 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:45.311280 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.311346 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.311465 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122147,"prepare.queue_time_us":610146,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":3514,"raft.queue_time_us":116818,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":13527,"spinlock_wait_cycles":373280000,"thread_start_us":424187,"threads_started":3,"wal-append.queue_time_us":480737}
I20260709 13:15:45.313112 11477 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:45.314090 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1475) took 1171 ms. Trace:
I20260709 13:15:45.280196 11476 raft_consensus.cc:2889] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.316603 11476 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:15:45.317931 13406 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.282s	user 0.005s	sys 0.005s
I20260709 13:15:45.314461 11542 rpcz_store.cc:276] 0709 13:15:44.142622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.479030 (+336408us) service_pool.cc:224] Handling call
0709 13:15:44.518448 (+ 39418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.740579 (+222131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.740623 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.740693 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:44.740737 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.313877 (+573140us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54001a7cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:45.313916 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.314039 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510287616}
I20260709 13:15:45.292908 13329 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:45.319751 13329 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:45.320104 13329 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:45.324081 13438 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:45.324522 13438 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.325385 13438 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.326624 13438 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:45.274572 13207 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.196787 13413 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.291656 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:45.291486 13423 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.292659 13392 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.328490 13399 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:45.328775 13392 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.329905 13423 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:45.292693 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 129071us
I20260709 13:15:45.331523 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:45.332762 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.334635 11442 leader_election.cc:400] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:45.291399 13198 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.335265 13198 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.335817 13411 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.336647 13412 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:45.292843 13378 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.268545 13055 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.271344 13113 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.334911 11442 leader_election.cc:403] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:45.295409 13200 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.295560 13335 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:45.296555 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.298282 13418 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.300735 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1461) took 1254 ms. Trace:
I20260709 13:15:45.279799 13373 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.357957 13373 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.358613 13198 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.304989 13174 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:45.330804 11468 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:45.225133 13381 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.360219 13373 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.360949 13335 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.362043 13381 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.362732 11477 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:45.341287 13055 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.363080 13335 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.356302 11470 rpcz_store.cc:276] 0709 13:15:44.046771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.048772 (+  2001us) service_pool.cc:224] Handling call
0709 13:15:44.049043 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:44.239063 (+190020us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:44.376929 (+137866us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.819921 (+442992us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:44.821179 (+  1258us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.821642 (+   463us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:44.821695 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.821797 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.300472 (+478675us) raft_consensus.cc:1685] finished
0709 13:15:45.300499 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.300666 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17793,"prepare.queue_time_us":471052,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":134116,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":21253,"thread_start_us":617335,"threads_started":3,"wal-append.queue_time_us":650094}
I20260709 13:15:45.364477 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:45.365103 11470 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:45.366168 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:45.341614 13452 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:45.342114 13113 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.340806 13258 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.369014 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.369462 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:45.360275 13174 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.370167 13174 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.370838 11474 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:45.371564 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:45.372150 11470 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:45.373471 11468 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:45.344404 11476 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:45.345517 13378 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.346313 13392 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.347363 11442 leader_election.cc:400] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 4.
I20260709 13:15:45.348124 13445 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.346527 13329 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.400345 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.400530 11470 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:45.353808 13423 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:45.357321 11473 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:15:45.340803 13351 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:45.340633 13444 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:45.378180 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.377403 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:45.409972 11514 leader_election.cc:336] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.838s (SENT)
I20260709 13:15:45.410231 11396 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:45.368942 13258 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.410667 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:45.400600 11474 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:45.399595 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.410025 11398 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:45.411674 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.407392 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.419373 13392 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:45.404074 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:45.408687 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.413441 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:45.420504 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 73261.2us
I20260709 13:15:45.422447 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:15:45.429155 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.419703 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:45.420872 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:45.440172 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:45.444283 11514 leader_election.cc:336] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.082s (SENT)
I20260709 13:15:45.456847 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.492952 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:45.451683 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.420123 13349 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.478255 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.430150 13399 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:45.419816 13392 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:45.493558 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:45.420300 11472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.007s	sys 0.015s
I20260709 13:15:45.456763 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.480880 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.508671 13452 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:45.509431 13451 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.510295 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.429279 13373 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:45.499960 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.428232 13418 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:45.433739 13423 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:45.500444 11475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.006s	sys 0.051s
W20260709 13:15:45.515466 11475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.006s	sys 0.051s
I20260709 13:15:45.517198 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1464) took 1465 ms. Trace:
W20260709 13:15:45.502969 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.007s	sys 0.023s
I20260709 13:15:45.466216 11393 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:45.502468 11396 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:45.444222 13407 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:45.512101 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.456387 13351 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:45.494534 13348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.025s	sys 0.038s
I20260709 13:15:45.481256 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:45.512483 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 68243.3us
W20260709 13:15:45.507220 11472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.016s
W20260709 13:15:45.520567 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.007s	sys 0.023s
I20260709 13:15:45.513638 11395 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:45.499511 13207 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:15:45.514758 13452 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:45.434478 13200 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:15:45.519868 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:45.578127 11473 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:45.521514 13350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.004s	sys 0.031s
I20260709 13:15:45.502079 13399 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:45.467373 13160 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:15:45.516062 13423 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.535408 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:45.586292 11469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.006s	sys 0.025s
I20260709 13:15:45.542657 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:45.456147 13445 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:45.575345 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.519554 13420 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:45.506820 13349 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.511718 13307 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:15:45.540536 13348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.026s	sys 0.038s
W20260709 13:15:45.532464 11548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.288s	user 0.004s	sys 0.025s
W20260709 13:15:45.596210 11469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.006s	sys 0.025s
I20260709 13:15:45.585208 11401 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:15:45.586450 12118 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.008s	sys 0.028s
I20260709 13:15:45.594667 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.515919 13418 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.460794 13174 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.517638 11475 rpcz_store.cc:276] 0709 13:15:44.051813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.741903 (+690090us) service_pool.cc:224] Handling call
0709 13:15:44.742216 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.746699 (+  4483us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:44.749171 (+  2472us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.516802 (+767631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.516861 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.516903 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.516990 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.517136 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2180,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":80,"thread_start_us":767273,"threads_started":1}
W20260709 13:15:45.605180 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62624.7us
I20260709 13:15:45.586258 13188 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:45.639127 13188 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.639895 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.640504 11399 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:45.641156 13188 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.612548 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.566967 13407 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:45.584437 13350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.004s	sys 0.032s
I20260709 13:15:45.598088 12855 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.624301 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:45.633155 12118 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.008s	sys 0.028s
W20260709 13:15:45.596637 13305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.393s	user 0.002s	sys 0.001s
I20260709 13:15:45.636399 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1470) took 1572 ms. Trace:
I20260709 13:15:45.645358 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.598078 13423 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:45.693266 13448 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.052s	sys 0.077s
W20260709 13:15:45.585928 13402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.011s	sys 0.013s
W20260709 13:15:45.669523 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45320.9us
W20260709 13:15:45.633442 11548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.006s	sys 0.026s
I20260709 13:15:45.683559 11400 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:45.686594 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.688104 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:45.643091 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:45.735368 11475 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:45.735483 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 92501.1us
I20260709 13:15:45.689790 11397 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:45.623848 13411 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:45.689455 12855 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.692162 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:45.692425 13174 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:45.643215 13352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.003s	sys 0.021s
I20260709 13:15:45.644119 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:45.754773 13352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.003s	sys 0.022s
I20260709 13:15:45.633885 13391 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.254s
I20260709 13:15:45.594751 13399 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:45.694247 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.017s	sys 0.057s
W20260709 13:15:45.757306 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.017s	sys 0.057s
W20260709 13:15:45.757622 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 765007us
I20260709 13:15:45.625524 13351 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:45.595366 12086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.004s	sys 0.008s
W20260709 13:15:45.760151 12086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.006s	sys 0.010s
W20260709 13:15:45.763078 13396 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.009s	sys 0.016s
W20260709 13:15:45.763437 13396 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.009s	sys 0.016s
I20260709 13:15:45.767781 13407 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.711077 11473 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:15:45.711476 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.711448 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1459) took 1594 ms. Trace:
W20260709 13:15:45.668807 13305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.507s	user 0.007s	sys 0.008s
I20260709 13:15:45.646277 13455 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:45.643091 13307 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:45.689883 13329 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:45.710646 13402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.011s	sys 0.014s
I20260709 13:15:45.695760 13413 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.598179 13392 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.731431 11442 leader_election.cc:400] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
I20260709 13:15:45.793268 11442 leader_election.cc:403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:15:45.707504 11543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.010s	sys 0.041s
W20260709 13:15:45.794025 11543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.011s	sys 0.042s
W20260709 13:15:45.723708 13448 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.054s	sys 0.079s
I20260709 13:15:45.732867 11395 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
W20260709 13:15:45.696033 13425 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.011s	sys 0.009s
I20260709 13:15:45.713580 13388 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.712997 13445 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:45.742548 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.747514 13373 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:15:45.726904 12883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.012s	sys 0.012s
W20260709 13:15:45.749245 13457 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:15:45.749564 12855 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.752257 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.758477 13160 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:45.774235 11548 rpcz_store.cc:276] 0709 13:15:44.117473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.118285 (+   812us) service_pool.cc:224] Handling call
0709 13:15:44.118608 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.118843 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:44.244740 (+125897us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.711096 (+1466356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.711155 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.711203 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.711274 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.711389 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":259771,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":3511,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9644,"thread_start_us":1578928,"threads_started":2}
I20260709 13:15:45.776314 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:45.782795 11366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.029s	sys 0.043s
W20260709 13:15:45.830200 11366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.029s	sys 0.043s
I20260709 13:15:45.746063 13286 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:45.806710 13388 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.833282 13286 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.833837 13286 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.741153 13411 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.835671 13411 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:45.839363 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63162.3us
W20260709 13:15:45.797116 11545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.009s	sys 0.020s
W20260709 13:15:45.793032 11471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.012s	sys 0.016s
W20260709 13:15:45.801594 12122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.001s	sys 0.021s
I20260709 13:15:45.690613 11472 rpcz_store.cc:276] 0709 13:15:44.061065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.845560 (+784495us) service_pool.cc:224] Handling call
0709 13:15:44.845906 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.846204 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:44.847941 (+  1737us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.597929 (+749988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.604563 (+  6634us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.611456 (+  6893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.630637 (+ 19181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.633666 (+  3029us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5138,"prepare.queue_time_us":1197,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":19836,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":6694,"thread_start_us":719198,"threads_started":1,"wal-append.queue_time_us":844572}
I20260709 13:15:45.782883 13307 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.819473 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:45.809751 12883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.012s	sys 0.012s
I20260709 13:15:45.827642 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:45.826846 11397 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
W20260709 13:15:45.802765 13425 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.011s	sys 0.010s
I20260709 13:15:45.783139 13455 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:45.810056 13385 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:45.843144 12122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.001s	sys 0.021s
I20260709 13:15:45.844311 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:45.844314 11546 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 3: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:45.842463 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:45.848693 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:45.843007 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.016s
I20260709 13:15:45.855765 11551 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:45.856516 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1472) took 1795 ms. Trace:
I20260709 13:15:45.856912 11471 rpcz_store.cc:276] 0709 13:15:44.061598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.223332 (+1161734us) service_pool.cc:224] Handling call
0709 13:15:45.223661 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.223867 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:45.224478 (+   611us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.856225 (+631747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.856273 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.856311 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.856370 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.856466 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":179,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"thread_start_us":631452,"threads_started":1}
W20260709 13:15:45.843057 11545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.009s	sys 0.020s
I20260709 13:15:45.799641 13413 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.858752 13429 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:45.861070 13413 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:45.852811 13418 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:45.853569 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 135419us
I20260709 13:15:45.864459 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:45.866642 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:45.848397 13207 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:45.841171 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:45.810861 13445 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.848973 12117 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:45.868216 13113 raft_consensus.cc:2764] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:15:45.875830 13349 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:45.882305 13310 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.021s	sys 0.017s
W20260709 13:15:45.883409 13055 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.003s
I20260709 13:15:45.866938 13385 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.882707 13207 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:45.892203 12923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.020s	sys 0.019s
I20260709 13:15:45.901708 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:45.893244 13400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.984s	user 0.014s	sys 0.028s
I20260709 13:15:45.891561 13329 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3.
W20260709 13:15:45.894790 13310 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.021s	sys 0.017s
W20260709 13:15:45.908903 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 366889us
W20260709 13:15:45.909626 11514 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.143s (SENT)
W20260709 13:15:45.918169 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.916369 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:45.913641 13373 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.922914 13418 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:45.922803 13373 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.925920 13373 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:45.930879 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 75879.9us
W20260709 13:15:45.934525 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.937976 13461 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:45.938083 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 304) took 2348 ms. Trace:
I20260709 13:15:45.938781 13461 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.938954 12122 rpcz_store.cc:276] 0709 13:15:43.589955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.639648 (+ 49693us) service_pool.cc:224] Handling call
0709 13:15:43.684989 (+ 45341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.685221 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:43.978504 (+293283us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.260471 (+1281967us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b18002c3fe0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:15:45.937200 (+676729us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.937410 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.937465 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.937754 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.937841 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.937992 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284142,"prepare.run_cpu_time_us":193,"prepare.run_wall_time_us":522,"raft.queue_time_us":3318,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":4419,"spinlock_wait_cycles":4041472,"thread_start_us":773677,"threads_started":3}
W20260709 13:15:45.943882 11514 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.698s (SENT)
I20260709 13:15:45.948140 13373 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:45.949106 13373 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:45.953564 11548 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:45.960793 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:45.902817 12923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.021s	sys 0.020s
I20260709 13:15:45.888851 12883 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:15:45.906508 13400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.010s	user 0.014s	sys 0.028s
I20260709 13:15:45.882323 13455 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.882407 13307 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:45.993345 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:45.965286 13113 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:45.994279 11544 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:45.991977 13373 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:45.901016 13055 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:46.001420 13465 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:15:46.030854 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1474) took 1888 ms. Trace:
W20260709 13:15:46.032586 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 158582us
I20260709 13:15:46.035626 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.037410 11549 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:46.038455 11544 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:46.038648 11543 rpcz_store.cc:276] 0709 13:15:44.142359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.397123 (+254764us) service_pool.cc:224] Handling call
0709 13:15:44.397450 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.397674 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:45.794525 (+1396851us) log.cc:844] Serialized 14 byte log entry
0709 13:15:46.030434 (+235909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:46.030496 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:46.030554 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.030632 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.030785 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":458817,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":67,"raft.queue_time_us":1405,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2256,"thread_start_us":1550047,"threads_started":2}
I20260709 13:15:46.035110 13445 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.060232 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.064721 11471 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:46.064528 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.069841 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.073206 11544 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:46.031975 13207 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.077553 13388 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:46.080863 13464 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:15:46.083180 13055 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:46.085731 11427 heartbeater.cc:646] Failed to heartbeat to 127.10.3.62:34815 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.10.3.62:34815 timed out after 15.000s (SENT)
I20260709 13:15:46.086337 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1466) took 2034 ms. Trace:
I20260709 13:15:46.086756 11469 rpcz_store.cc:276] 0709 13:15:44.052399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.780399 (+728000us) service_pool.cc:224] Handling call
0709 13:15:44.780696 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:44.781036 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.781066 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:44.781554 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:15:45.632606 (+851052us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:45.632645 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.632712 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.086104 (+453392us) raft_consensus.cc:1685] finished
0709 13:15:46.086125 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.086279 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10043,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":850608,"threads_started":1,"wal-append.queue_time_us":1087156}
I20260709 13:15:46.089354 13399 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.068224 13113 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.052723 13423 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.122758 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:46.123588 11471 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:46.119211 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.130275 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.115587 13425 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.136719 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:46.146853 11514 leader_election.cc:336] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.191s (SENT)
I20260709 13:15:46.147539 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:46.148327 11474 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:46.149498 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:46.150149 13481 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:15:46.150259 11397 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:46.152160 13423 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.153172 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.153261 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.153882 11396 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:46.155321 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.155496 11473 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:46.154827 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.156682 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.158895 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.159277 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.161734 13421 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:46.162125 13421 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:46.162523 13421 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.163507 13474 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:46.132786 13420 raft_consensus.cc:2764] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:15:46.164139 11546 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:46.139247 12855 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:46.165894 12855 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.166270 12855 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.168748 13411 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.170452 13421 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.127070 13305 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:46.136776 13385 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.165522 11546 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:15:46.188133 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:46.194196 11514 leader_election.cc:336] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.672s (SENT)
I20260709 13:15:46.197719 13402 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.203658 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:46.194923 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64629.1us
I20260709 13:15:46.203061 13469 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:15:46.220360 11395 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:15:46.171950 13495 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:15:46.223266 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:46.219379 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.223982 11474 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:15:46.172353 13482 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:15:46.229768 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:46.226634 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53201.6us
I20260709 13:15:46.170960 13411 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.186383 13421 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.186913 13385 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:46.249881 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 55729us
W20260709 13:15:46.251919 11514 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.162s (SENT)
I20260709 13:15:46.256283 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:46.256606 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.257678 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:46.222818 13399 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:46.171499 13305 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.175544 13425 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.263672 11474 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:46.295504 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:46.296025 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44431.6us
I20260709 13:15:46.296232 11399 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:46.298157 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:46.301617 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49739.6us
W20260709 13:15:46.315471 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 666848us
I20260709 13:15:46.261696 13391 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:46.321317 11514 leader_election.cc:336] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.046s (SENT)
I20260709 13:15:46.321911 13388 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:46.322417 13388 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.323521 11395 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:46.324496 13388 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.325302 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.327456 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.328078 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:46.254809 13448 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
W20260709 13:15:46.333236 13479 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
W20260709 13:15:46.333478 13503 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:15:46.333659 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:46.334914 13500 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:15:46.288024 13411 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.343540 13493 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.344342 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.254288 13385 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:46.366653 13494 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 13:15:46.390897 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71569.2us
I20260709 13:15:46.392132 13512 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.319628 13286 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.396533 13512 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.397141 13512 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:46.397535 13508 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:15:46.398747 13378 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.999s	user 0.003s	sys 0.009s
I20260709 13:15:46.401454 11548 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
W20260709 13:15:46.347093 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.331960 13391 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.400345 11402 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:46.407706 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.410679 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1461) took 2280 ms. Trace:
W20260709 13:15:46.371141 13492 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:46.412871 11545 rpcz_store.cc:276] 0709 13:15:44.130741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.131057 (+   316us) service_pool.cc:224] Handling call
0709 13:15:44.131393 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:44.149221 (+ 17828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.149255 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:44.155397 (+  6142us) log.cc:844] Serialized 26 byte log entry
0709 13:15:45.233073 (+1077676us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b180035bba0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:45.858501 (+625428us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:45.858566 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.858666 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.410332 (+551666us) raft_consensus.cc:1685] finished
0709 13:15:46.410351 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.410596 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19545,"raft.queue_time_us":20803,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":4789504,"thread_start_us":625259,"threads_started":1,"wal-append.queue_time_us":1826288}
I20260709 13:15:46.314529 13501 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:15:46.341202 13349 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:46.426455 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.394850 13425 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.430781 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.448972 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:46.451884 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.414024 13391 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.458770 13351 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.400305 13493 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.463454 13286 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:46.464677 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 134318us
I20260709 13:15:46.423527 13113 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.471427 11399 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:46.465402 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.472975 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.473595 11395 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.476176 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:46.476792 11474 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:46.478950 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.400559 13460 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:15:46.489675 11395 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:46.455441 13420 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.491643 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:46.451262 13305 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.491331 13286 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.493920 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:46.499610 13490 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:15:46.502456 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.479454 13310 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:46.508674 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.510219 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.511022 12119 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:46.511443 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:46.512732 13385 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.513327 13512 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:46.516000 13310 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:46.517270 13310 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.518865 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 305) took 1825 ms. Trace:
I20260709 13:15:46.519280 12118 rpcz_store.cc:276] 0709 13:15:44.693078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.693213 (+   135us) service_pool.cc:224] Handling call
0709 13:15:44.702682 (+  9469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.684051 (+981369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.684078 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:45.692409 (+  8331us) log.cc:844] Serialized 26 byte log entry
0709 13:15:46.038521 (+346112us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:46.038560 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.038641 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.518625 (+479984us) raft_consensus.cc:1685] finished
0709 13:15:46.518647 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.518808 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":309502,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":1311705,"threads_started":2,"wal-append.queue_time_us":561697}
W20260709 13:15:46.522539 13504 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 13:15:46.482653 12855 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.563187 13418 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.563997 13418 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.564594 12855 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.566455 13418 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.571815 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.575006 13113 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:46.547829 13420 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.509222 13286 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.609232 13286 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:46.683255 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 111514us
I20260709 13:15:46.683811 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.689343 11549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:46.690210 11549 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:46.690548 11549 raft_consensus.cc:614] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:46.677842 13286 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.703699 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 360) took 1017 ms. Trace:
I20260709 13:15:46.704165 12117 rpcz_store.cc:276] 0709 13:15:45.686376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.690204 (+  3828us) service_pool.cc:224] Handling call
0709 13:15:46.703514 (+1013310us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b54005ff0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:46.703626 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7474944}
I20260709 13:15:46.705627 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.697207 13286 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.707196 13420 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:46.707732 13420 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:46.708292 13286 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.708607 13420 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.711704 13455 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:46.704774 13391 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.716223 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.733196 11397 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:46.717907 13391 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:46.715075 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.734424 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.722069 11549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:46.735921 11549 raft_consensus.cc:606] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:46.736281 11549 raft_consensus.cc:614] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:46.726127 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:46.737226 11470 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:46.737970 13522 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:46.738407 13522 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.738734 13522 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.741057 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.742017 13055 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:46.743433 13055 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:46.743778 13286 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:46.744241 13286 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:46.744421 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.745430 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:46.745085 13286 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.745635 13159 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:46.746233 13159 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.747430 13521 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:46.748520 13159 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.748908 13521 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.749866 11549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:46.750504 11549 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:46.750859 11549 raft_consensus.cc:614] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:46.695318 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.726460 11474 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.751806 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:46.742779 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:46.728050 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:46.732362 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.733222 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:46.757280 11473 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.753911 11516 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.759707 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.764588 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.766266 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:46.781275 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.746471 13286 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.817144 11470 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:46.820273 11474 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:15:46.820824 11474 pending_rounds.cc:85] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 1
I20260709 13:15:46.821717 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:46.825079 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:46.826304 11470 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:46.761714 13391 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:46.754664 13113 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.828610 11473 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:46.764348 13418 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:46.821123 11474 pending_rounds.cc:107] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:46.831019 11474 raft_consensus.cc:2889] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:46.764748 13425 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.818326 13373 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:46.802539 13385 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.802899 13055 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.822916 13521 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:46.830606 13391 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:46.845237 11471 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:46.833382 13418 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.838936 13373 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.846652 13391 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:46.859812 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66452.9us
I20260709 13:15:46.841761 13521 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.834324 13113 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:46.855806 13373 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:46.864336 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 97737.8us
I20260709 13:15:46.849280 13418 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:46.859709 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.861657 13521 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:46.867209 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:46.917196 13113 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:46.918030 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50960.8us
W20260709 13:15:46.918381 11514 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.600s (SENT)
I20260709 13:15:46.919305 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:46.922009 11514 leader_election.cc:336] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.619s (SENT)
I20260709 13:15:46.922506 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.916786 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:46.927356 13055 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:46.928833 13535 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:46.929747 13535 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:46.931162 13349 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.931759 13349 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.932137 13286 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:46.932761 13392 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:46.932936 13286 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:46.933416 13286 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:46.921878 13420 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:46.926184 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:46.942710 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.946400 13534 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:46.947448 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.949394 13493 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:46.951102 13456 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:46.951540 13456 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:46.951767 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.952607 11395 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:46.952390 13456 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.953413 13456 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:46.956738 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.959486 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.960232 11397 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:46.961813 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.968418 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.969424 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:46.971585 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:46.972510 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.973644 11393 raft_consensus.cc:2410] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:15:46.974032 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50688.3us
I20260709 13:15:46.974735 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.976199 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.978820 13493 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.980036 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:46.980516 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:46.980194 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.981976 11393 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:46.982715 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:46.985424 11551 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:46.988529 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.990340 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.991462 11398 raft_consensus.cc:2393] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:46.992610 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:46.983455 13493 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:46.983580 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:46.994189 13493 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:46.995378 13493 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:46.984198 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:46.999888 11546 raft_consensus.cc:2410] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:47.002876 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.005043 11516 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:47.008759 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.009274 11399 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:47.010701 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.011446 11546 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.934846 13286 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:47.017810 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66963.6us
I20260709 13:15:47.049450 11474 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:46.966012 13534 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:47.051578 13446 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:47.051841 13534 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.052261 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:47.057919 11551 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:46.945282 13420 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.061425 11395 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
W20260709 13:15:47.062824 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 141590us
I20260709 13:15:47.062951 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:47.063484 11514 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.619s (SENT)
I20260709 13:15:47.063874 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.072577 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.077271 11398 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.082793 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.083648 11551 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:47.082787 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:47.085793 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.059644 13055 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:47.070735 13286 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:47.088886 13521 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.088017 13055 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.093237 13541 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.108726 13425 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.109282 13425 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.113919 13373 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.116962 13349 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.120838 11393 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:47.120435 13522 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.125100 13391 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.133680 11397 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:47.124828 11551 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:15:47.076545 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.139411 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.147898 13349 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:47.148123 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.148922 11472 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.149302 13391 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:47.149808 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.150359 11472 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.152510 11549 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:47.154249 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.155031 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.136937 13522 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:47.173504 12117 raft_consensus.cc:2368] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6.
I20260709 13:15:47.142421 12112 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:47.187839 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.179366 13373 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:47.206017 13418 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.207840 13420 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:47.210979 13491 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.069s	user 0.002s	sys 0.003s
I20260709 13:15:47.221017 11544 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:47.223601 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.224318 11544 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.214128 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.228245 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.216773 13489 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.082s	user 0.000s	sys 0.011s
I20260709 13:15:47.229964 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 309) took 1830 ms. Trace:
I20260709 13:15:47.230317 12113 rpcz_store.cc:276] 0709 13:15:45.400046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.411730 (+ 11684us) service_pool.cc:224] Handling call
0709 13:15:45.481399 (+ 69669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.481850 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.481885 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:45.499981 (+ 18096us) log.cc:844] Serialized 26 byte log entry
0709 13:15:45.913556 (+413575us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.914091 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.915312 (+  1221us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:47.216161 (+1300849us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:47.229663 (+ 13502us) raft_consensus.cc:1685] finished
0709 13:15:47.229699 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.229905 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":133712768,"thread_start_us":327294,"threads_started":1,"wal-append.queue_time_us":551825}
I20260709 13:15:47.232489 11470 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:47.233402 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.234424 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.237354 11398 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:47.237984 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.255398 11393 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:47.256090 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.238867 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.257747 11401 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.259879 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.262946 13373 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.263551 13373 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.241019 13418 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:47.228626 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.265237 11542 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:47.265316 11398 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:47.265450 13373 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:47.266170 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.266978 11542 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:47.269150 12113 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:47.269907 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.271037 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.220074 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.243309 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 206) took 1463 ms. Trace:
I20260709 13:15:47.272146 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.271958 11400 rpcz_store.cc:276] 0709 13:15:45.779920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.780112 (+   192us) service_pool.cc:224] Handling call
0709 13:15:45.780509 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.887780 (+107271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.887823 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.887885 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.887929 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.243027 (+1355098us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400230854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:47.243073 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.243228 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246101888}
I20260709 13:15:47.273066 11475 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.273216 11401 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:47.272540 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.273878 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.244302 13486 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.176s	user 0.004s	sys 0.015s
I20260709 13:15:47.274811 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.275490 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1496) took 1933 ms. Trace:
I20260709 13:15:47.276007 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.275837 11476 rpcz_store.cc:276] 0709 13:15:45.342483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.401551 (+ 59068us) service_pool.cc:224] Handling call
0709 13:15:45.402196 (+   645us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:45.409937 (+  7741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.409967 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:45.461668 (+ 51701us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:15:45.487856 (+ 26188us) log.cc:844] Serialized 566 byte log entry
0709 13:15:45.890395 (+402539us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.890458 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.890549 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:47.243567 (+1353018us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:47.275269 (+ 31702us) raft_consensus.cc:1685] finished
0709 13:15:47.275290 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.275439 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7805,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":7397,"spinlock_wait_cycles":11681536,"thread_start_us":368760,"threads_started":1,"wal-append.queue_time_us":526526}
I20260709 13:15:47.278075 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.276122 11393 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:47.245240 11546 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
W20260709 13:15:47.243741 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.246961 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.246654 11472 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:47.251365 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.279897 11397 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.253471 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.281137 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.282074 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.254091 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.240044 11395 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:47.274008 11400 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:47.287366 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.280059 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:47.249405 11551 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.283706 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:47.266170 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.288769 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.289922 11473 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.290572 11397 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:47.302089 11544 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:47.285622 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.291510 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.293368 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 229898us
I20260709 13:15:47.319384 11476 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:15:47.319507 11514 leader_election.cc:336] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.629s (SENT)
I20260709 13:15:47.320140 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:47.321208 11514 leader_election.cc:336] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.674s (SENT)
I20260709 13:15:47.321601 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:47.295218 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.284610 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.322536 11514 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.685s (SENT)
I20260709 13:15:47.322898 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:47.323098 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.323962 11472 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.334354 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.336469 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.336586 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.337600 11473 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.338667 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.353977 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.337476 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.354975 11472 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:47.355716 11470 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.357401 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.279637 13385 raft_consensus.cc:2764] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:15:47.358062 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.359349 11546 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:47.359583 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.355865 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.365583 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.366530 11397 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:47.361299 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.362637 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.367637 11473 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.368366 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.373189 11477 raft_consensus.cc:2368] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:15:47.374469 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.351872 13411 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.375783 11400 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.375111 11471 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:47.376974 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.377935 11472 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.378931 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:47.380443 11546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:47.380960 11546 raft_consensus.cc:606] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:47.381274 11546 raft_consensus.cc:614] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:47.381986 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.382648 11470 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.369735 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:47.378785 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.327630 13055 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:47.337348 13373 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:47.299057 13352 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:47.383910 11544 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:47.438606 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.440053 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.384703 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.339293 13467 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:47.356884 13522 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:15:47.443264 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.291899 13458 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.148s
I20260709 13:15:47.286147 13438 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 13:15:47.458349 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:47.459439 11514 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.708s (SENT)
I20260709 13:15:47.493642 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:47.460244 11397 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:47.491499 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:47.495970 11514 leader_election.cc:336] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.729s (SENT)
I20260709 13:15:47.497187 11472 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.375494 13461 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:47.351600 13498 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:15:47.499442 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:47.498631 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 434249us
I20260709 13:15:47.499146 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.384670 11476 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:47.462771 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:47.467913 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.471462 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.473531 11469 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.475586 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:47.357600 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.476821 13467 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:47.462745 13352 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:47.486223 13412 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:47.487108 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:47.489030 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 179) took 1185 ms. Trace:
I20260709 13:15:47.491420 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.503414 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.514463 11476 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:47.361840 13335 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:47.505085 11477 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 2: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.512857 11545 rpcz_store.cc:276] 0709 13:15:46.303414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:46.449834 (+146420us) service_pool.cc:224] Handling call
0709 13:15:46.450197 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:46.464977 (+ 14780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:46.465004 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:46.465455 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:15:46.525998 (+ 60543us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:46.526034 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.488565 (+962531us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54000b2a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:47.488615 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:47.488675 (+    60us) raft_consensus.cc:1685] finished
0709 13:15:47.488695 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.488957 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5546,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":33094912,"thread_start_us":60186,"threads_started":1,"wal-append.queue_time_us":95696}
I20260709 13:15:47.516209 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.517556 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.510206 11471 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:47.518911 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.511891 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.520169 11469 raft_consensus.cc:2376] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4. Re-sending same reply.
I20260709 13:15:47.513481 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.521384 11473 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.521718 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.524166 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.524909 11473 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:47.525875 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.527312 11470 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.528295 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.532238 11470 raft_consensus.cc:2368] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:47.532059 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.532353 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.533041 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.533479 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.533946 11470 raft_consensus.cc:2368] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:47.535006 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.535328 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.536666 11471 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:47.537936 11476 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:47.538388 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:47.539996 11476 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:47.541134 11476 raft_consensus.cc:1217] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:15:47.542047 11476 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:47.544781 11473 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
I20260709 13:15:47.545785 11471 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:47.534937 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:47.357054 12923 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 13:15:47.559399 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:47.559752 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 91907.8us
I20260709 13:15:47.560323 11469 raft_consensus.cc:2368] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:47.560796 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.561650 11470 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:15:47.549698 11475 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:47.561870 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:47.504261 13548 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:47.512460 13412 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:47.523108 13553 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.564538 13113 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:47.512053 13498 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.563337 13429 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.563701 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:47.585516 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.564015 13551 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:47.578644 13553 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.595881 13352 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.576721 13412 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.581737 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1495) took 1761 ms. Trace:
I20260709 13:15:47.616144 11550 rpcz_store.cc:276] 0709 13:15:45.819964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.820087 (+   123us) service_pool.cc:224] Handling call
0709 13:15:45.841999 (+ 21912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.962098 (+120099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.962147 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.962201 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.962273 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.581431 (+1619158us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003eadd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:47.581492 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.581666 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275685248}
W20260709 13:15:47.565101 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 413139us
I20260709 13:15:47.770326 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1519) took 1037 ms. Trace:
I20260709 13:15:47.669195 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:47.667065 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 313132us
I20260709 13:15:47.670610 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 167) took 1715 ms. Trace:
I20260709 13:15:47.825999 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.833220 11468 rpcz_store.cc:276] 0709 13:15:46.733375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:46.822868 (+ 89493us) service_pool.cc:224] Handling call
0709 13:15:47.769638 (+946770us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:47.769816 (+   178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:47.769986 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:47.770010 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:47.770047 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:47.770077 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.770129 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.770257 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17689344}
I20260709 13:15:47.834571 11548 rpcz_store.cc:276] 0709 13:15:45.954935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.955506 (+   571us) service_pool.cc:224] Handling call
0709 13:15:45.956020 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:46.401865 (+445845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.670351 (+1268486us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54000b2f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:47.670549 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023981440}
I20260709 13:15:47.818264 11399 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:47.836024 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.602753 13553 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:47.839918 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:47.840296 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 171193us
I20260709 13:15:47.813071 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.681366 12115 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:47.840754 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:47.844934 11546 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:15:47.691833 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:47.720429 13561 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:15:47.662597 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:47.846496 11547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:47.851583 11547 raft_consensus.cc:606] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:47.851926 11547 raft_consensus.cc:614] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:47.845747 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:47.759219 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 197385us
I20260709 13:15:47.621338 13113 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:47.728474 13307 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:47.620249 13551 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:47.858068 11515 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:47.856199 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.726329 13420 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.837801 13507 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:47.860872 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.713191 13458 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:47.796041 13564 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.752893 13498 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:47.615518 13429 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.868202 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:47.846426 13207 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:15:47.869803 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:47.866755 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:47.843751 13560 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.863895 13553 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.855057 13554 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:15:47.863359 13411 raft_consensus.cc:2764] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:47.850689 13541 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:15:47.863814 13507 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:47.874639 13570 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:47.889062 13385 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:47.865860 13113 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:47.867416 13458 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:47.868934 13462 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.869280 13535 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:47.824903 13412 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.595929 13467 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:47.937788 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 77288.2us
W20260709 13:15:47.940685 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 94994.4us
I20260709 13:15:47.899641 13551 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.942215 13425 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:47.881976 13564 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:47.943989 13113 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:47.946388 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:47.900360 13553 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:47.871129 13420 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:47.925980 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1032) took 2474 ms (client timeout 1919 ms). Trace:
I20260709 13:15:47.915102 13554 consensus_queue.cc:1243] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:15:47.953039 11402 rpcz_store.cc:269] 0709 13:15:45.451237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.451368 (+   131us) service_pool.cc:224] Handling call
0709 13:15:47.925752 (+2474384us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400125cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:47.925890 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077313920}
I20260709 13:15:47.944069 13200 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:47.923828 13535 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:47.951817 12115 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:47.964003 13200 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:47.946108 13564 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:47.978943 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110798us
I20260709 13:15:48.064661 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:48.072371 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:48.093034 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 154477us
W20260709 13:15:48.094496 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.967029 13535 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:48.100227 13568 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:15:47.963661 13553 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:48.101483 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 252196us
I20260709 13:15:48.094942 13573 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:15:48.097440 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.935878 13462 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:47.940395 13507 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.096973 13564 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:48.112380 11514 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.683s (SENT)
I20260709 13:15:48.112972 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.099607 13412 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:48.127692 11514 leader_election.cc:336] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.765s (SENT)
I20260709 13:15:48.128181 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.132875 13572 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:48.133394 13553 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.137367 13438 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.137948 13438 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.119856 13200 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.139911 13200 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.140494 13438 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.150534 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:48.110515 13573 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:48.154156 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.112097 13564 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.128090 13521 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.130292 13055 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:48.138684 13569 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:15:48.152602 13396 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.097613 13458 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.162431 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1585) took 1146 ms. Trace:
I20260709 13:15:48.170953 11551 rpcz_store.cc:276] 0709 13:15:47.015923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.139082 (+123159us) service_pool.cc:224] Handling call
0709 13:15:48.162182 (+1023100us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540041c8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:48.162353 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245683968}
I20260709 13:15:48.131770 13550 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:15:48.159307 13564 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:48.177711 13575 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:48.178227 13429 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:48.181182 13458 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:48.188005 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78039us
W20260709 13:15:48.191782 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.183140 13055 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:48.187539 13420 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.198081 13055 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:48.191414 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 124398us
I20260709 13:15:48.199028 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.199077 13055 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.199904 13420 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.200481 13055 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:48.200809 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:48.201581 13420 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.201901 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.203390 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:48.209921 11299 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 214) took 16461 ms (client timeout 15000 ms). Trace:
I20260709 13:15:48.210937 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:48.185215 13575 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:48.226248 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:48.263080 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70520.9us
W20260709 13:15:48.269842 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47882.2us
I20260709 13:15:48.223119 13373 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:48.273643 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47482.6us
I20260709 13:15:48.275177 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.264276 13570 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.222115 13420 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.259222 13113 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.268576 13533 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:48.217264 11299 rpcz_store.cc:269] 0709 13:15:31.748156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.802742 (+ 54586us) service_pool.cc:224] Handling call
0709 13:15:32.083560 (+280818us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:48.209837 (+16126277us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:43.892407 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:43.893135 (+   728us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:43.893199 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:15:43.893218 (+    19us) tablet.cc:662] Decoding operations
0709 13:15:43.893853 (+   635us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:43.893945 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:43.893977 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:15:43.894452 (+   475us) tablet.cc:701] Row locks acquired
0709 13:15:43.894520 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:15:43.894690 (+   170us) write_op.cc:270] Start()
0709 13:15:43.895244 (+   554us) write_op.cc:276] Timestamp: P: 1783602943894621 usec, L: 0
0709 13:15:43.895267 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:43.896026 (+   759us) log.cc:844] Serialized 1285 byte log entry
0709 13:15:45.208457 (+1312431us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da680f0ac 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b
0709 13:15:46.399301 (+1190844us) op_driver.cc:464] REPLICATION: finished
0709 13:15:46.695198 (+295897us) write_op.cc:301] APPLY: starting
0709 13:15:46.696487 (+  1289us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:46.700371 (+  3884us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:46.700396 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:46.700867 (+   471us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:46.701038 (+   171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:15:46.701085 (+    47us) write_op.cc:312] APPLY: finished
0709 13:15:46.701280 (+   195us) log.cc:844] Serialized 35 byte log entry
0709 13:15:46.701449 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:46.701536 (+    87us) write_op.cc:454] Released schema lock
0709 13:15:46.701580 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:46.735389 (+ 33809us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:46.735418 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":117391,"reported_tablets":9,"spinlock_wait_cycles":29723008,"child_traces":[["op",{"apply.queue_time_us":278374,"apply.run_cpu_time_us":2028,"apply.run_wall_time_us":49567,"num_ops":3,"prepare.queue_time_us":483144,"prepare.run_cpu_time_us":45685,"prepare.run_wall_time_us":2102343,"raft.queue_time_us":254777,"raft.run_cpu_time_us":11935,"raft.run_wall_time_us":207626,"replication_time_us":2440719,"spinlock_wait_cycles":96289408,"thread_start_us":2516860,"threads_started":4,"wal-append.queue_time_us":2104196,"wal-append.run_cpu_time_us":76801,"wal-append.run_wall_time_us":1699240}]]}
W20260709 13:15:48.287209 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:48.288690 11514 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.199s (SENT)
I20260709 13:15:48.289314 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.290244 13429 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.269568 13581 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.258046 13335 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:15:48.297642 12113 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:48.297901 13420 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.298403 13420 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.299576 13575 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.300455 13420 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:48.300606 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.300992 13420 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.301647 13550 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.302153 13550 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.302845 13412 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:48.304273 13550 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:48.244380 13399 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:15:48.307349 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.272357 13455 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:15:48.301483 13420 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.318655 11394 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:48.223251 13573 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.325075 13591 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.326004 13591 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.277801 13373 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:48.331609 11514 leader_election.cc:336] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.099s (SENT)
I20260709 13:15:48.283665 13570 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:48.339915 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.361657 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.363976 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:48.354001 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.332166 13420 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.317780 13551 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:48.356036 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77539.1us
I20260709 13:15:48.294193 13533 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.374974 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:48.377604 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.377530 11468 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.382203 13595 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:48.313130 13581 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.388545 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.386909 13373 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:48.369318 13462 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:48.389999 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 73429.2us
I20260709 13:15:48.368816 13591 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.369716 13553 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.357566 13550 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:15:48.377030 13420 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.396986 13591 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.399214 13506 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:48.399102 13594 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.377118 13592 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.382928 13595 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:48.414240 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.391181 13396 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:48.416095 13592 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.420806 13592 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.408609 13506 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:48.424119 13506 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.401190 13591 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.430949 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.431284 13506 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.408795 13553 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.432598 13581 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:48.433028 13553 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.438350 11396 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:48.440033 11468 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:48.383314 13551 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.415673 13594 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.439198 13407 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.447697 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:48.447077 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:48.469292 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.499125 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.505900 11474 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:48.502206 11471 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:48.503162 12117 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:48.463758 11468 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:15:48.466013 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.410621 13420 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.396840 13589 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 13:15:48.508162 12112 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:15:48.446818 13396 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:48.524698 13586 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:15:48.533914 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 258807us
I20260709 13:15:48.510677 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 361) took 1032 ms. Trace:
I20260709 13:15:48.528796 13595 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:48.530444 13533 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.522796 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:48.503460 13407 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.528425 13594 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:48.535180 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.522760 12112 pending_rounds.cc:85] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 6
I20260709 13:15:48.548519 13462 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:48.551054 13462 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:48.551729 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.552011 13462 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.552945 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.535684 12113 rpcz_store.cc:276] 0709 13:15:47.478140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.480235 (+  2095us) service_pool.cc:224] Handling call
0709 13:15:47.480587 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.298104 (+817517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.495719 (+197615us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54005ff0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:48.509745 (+ 14026us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875878144}
I20260709 13:15:48.550480 12112 pending_rounds.cc:107] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:15:48.555992 12112 raft_consensus.cc:2889] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:48.556291 13507 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.557056 13591 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.557808 13591 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:48.558308 13591 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:48.559180 13591 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.563165 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:48.563920 13551 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:48.564461 13587 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:15:48.564546 13551 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.566732 13553 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:48.567859 13581 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:48.568269 13581 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:48.569118 13581 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.570429 13396 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:48.570508 13581 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.574648 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1514) took 1317 ms. Trace:
I20260709 13:15:48.577324 11543 rpcz_store.cc:276] 0709 13:15:47.257663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.328346 (+ 70683us) service_pool.cc:224] Handling call
0709 13:15:48.162918 (+834572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:48.163377 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.163416 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:48.170530 (+  7114us) log.cc:844] Serialized 26 byte log entry
0709 13:15:48.312424 (+141894us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:48.312450 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.312495 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:48.572920 (+260425us) raft_consensus.cc:1685] finished
0709 13:15:48.572940 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:48.574407 (+  1467us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b540041c8d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:48.574583 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":575,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":8223,"spinlock_wait_cycles":1918281088,"thread_start_us":124943,"threads_started":1,"wal-append.queue_time_us":190080}
I20260709 13:15:48.575402 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.550840 13599 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:48.562430 13564 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.595958 13420 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.596616 13564 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:48.588418 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.597332 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.589587 11399 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:48.590655 11402 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:48.592058 11397 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:48.594753 13570 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.558827 13535 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.601297 13535 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:48.593472 13599 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:48.593731 11401 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
W20260709 13:15:48.594096 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.603520 13599 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.594822 13594 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.606660 11543 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: [2.5-5.6]
I20260709 13:15:48.584568 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.609769 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:48.609854 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.612365 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.612695 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.614967 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.615406 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.618150 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.618875 11474 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:48.619771 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.623376 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.621654 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.626789 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:48.601503 13593 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:15:48.629348 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.629971 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.630213 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.631640 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.633288 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.633772 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.629681 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.634681 11470 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:48.634919 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.635959 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:48.637358 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52868.7us
I20260709 13:15:48.639945 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.640708 11470 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:48.643194 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.643652 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.643833 11468 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:48.645098 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.646974 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:48.647361 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.647743 11477 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:48.648566 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.648633 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.649434 11473 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:48.649195 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:48.650466 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 265619us
I20260709 13:15:48.650671 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.654405 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.655232 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.655772 11473 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:48.658448 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.661459 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.666018 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:48.666253 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.667217 11477 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:48.667919 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:48.621564 13581 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.671195 13581 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.632551 13589 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.674862 11470 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:15:48.674732 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.635215 12923 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:48.643389 13533 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:48.680256 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.644155 13551 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:48.643028 13594 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.680915 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:48.628198 13200 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:15:48.682878 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.648386 13564 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.683908 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.655079 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:48.665786 13535 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.689844 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.690075 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.692538 11470 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 2.
W20260709 13:15:48.691157 11514 leader_election.cc:336] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.496s (SENT)
I20260709 13:15:48.696288 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.696938 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:48.701951 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.704075 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.715701 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.716442 11470 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:48.717258 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.667088 13467 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 13:15:48.716468 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.718745 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.726478 11469 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:48.691812 13572 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.727831 11470 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
W20260709 13:15:48.727905 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 62152.1us
I20260709 13:15:48.729779 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:48.731128 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.734692 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:48.748370 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.683828 12923 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.701284 13535 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.756821 12923 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.716634 13570 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.681169 13589 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:48.758678 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 71649.5us
I20260709 13:15:48.758769 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.716881 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:48.778406 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.736454 13391 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:48.779675 13391 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.780043 13391 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:48.780411 13535 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:48.687774 13551 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.693583 13594 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.698021 13564 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.792701 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.793448 11398 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:48.716579 13550 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:15:48.764765 13541 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:48.790962 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:48.799350 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1504) took 1798 ms (client timeout 1523 ms). Trace:
I20260709 13:15:48.799614 11401 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:15:48.792188 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:48.799742 11396 rpcz_store.cc:269] 0709 13:15:47.002092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.002534 (+   442us) service_pool.cc:224] Handling call
0709 13:15:48.799181 (+1796647us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400348354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:48.799281 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3092758400}
I20260709 13:15:48.795693 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.800631 11545 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:15:48.789937 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 132381us
W20260709 13:15:48.796393 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.762830 13533 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.796587 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:48.819092 11544 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:48.819497 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54326.8us
I20260709 13:15:48.813616 11545 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:48.727211 13572 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.764948 13606 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.828339 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.831285 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:48.831863 11476 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:48.836454 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:48.797994 13588 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.400s	user 0.000s	sys 0.002s
I20260709 13:15:48.796767 13535 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.798445 13581 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:48.850927 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 92229.1us
I20260709 13:15:48.831032 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:48.834481 13522 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:48.823307 13595 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:48.862509 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.862545 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:48.866415 11544 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:48.871163 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:48.886870 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.830566 13533 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.896729 12116 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:48.898070 13533 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.871099 13373 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.899983 13589 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:48.911494 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:48.930303 11549 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:48.931622 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.893996 13573 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.893563 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:48.940497 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:48.953622 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:48.947953 12113 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:15:48.952936 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:48.908939 13535 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:15:48.892970 13595 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:48.959120 11469 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
W20260709 13:15:48.960925 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.946161 13373 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:48.979586 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 143656us
I20260709 13:15:48.986702 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:48.977648 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.988730 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.989476 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:48.993907 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:48.978893 13550 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:48.945205 13624 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 13:15:48.991528 13612 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:15:48.945443 13573 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:48.982553 13541 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.001233 13550 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.042795 13550 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:48.996639 13599 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.001780 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:49.056607 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54903.5us
I20260709 13:15:48.989041 13444 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.765272 13562 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:49.002592 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.032653 13458 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:49.013154 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.090533 13570 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:49.088382 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 85851.2us
I20260709 13:15:49.106314 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:49.013983 13620 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:15:49.009984 13621 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:15:49.017516 13615 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:15:48.998518 13617 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:49.010949 13626 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:15:49.013587 13616 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:15:49.010453 13605 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:15:49.107606 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.004597 13420 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.014529 13551 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:49.122805 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 169296us
I20260709 13:15:49.141111 13573 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.141232 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.121529 13617 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:49.016251 13564 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.090540 13476 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.147887 13476 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.018355 13594 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.017340 13595 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.017911 13572 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:49.087059 13624 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:49.130430 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.108003 13458 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.137167 13575 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.148255 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:49.149755 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:49.143334 13617 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.158390 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.097395 13444 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.017727 13391 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.150974 13541 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:49.148339 13476 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.169886 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.155279 13599 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.175500 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.177659 13373 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.179639 13535 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.180734 13438 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.181228 13438 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.184212 13438 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.185868 11543 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:49.189034 13444 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.161532 13594 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:49.190132 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.190248 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.191133 11542 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:15:49.191318 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:49.193590 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.194350 11542 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:49.165469 13572 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:49.201408 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.204874 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:49.156529 13550 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.209362 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:49.208508 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 100981us
I20260709 13:15:49.162040 13624 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.200623 13541 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.213590 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.191087 13631 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.165362 13575 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.204890 13572 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.228485 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.229794 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.232621 13564 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.234838 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.236306 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.237229 13595 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.238060 13595 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.238806 13595 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.248849 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:49.249708 11398 raft_consensus.cc:2376] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3. Re-sending same reply.
I20260709 13:15:49.250914 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.251765 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.252588 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.253101 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.253506 11400 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:49.254320 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.255338 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.256536 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.257153 11543 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:49.258072 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.259228 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.265112 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.213268 13438 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.275286 13438 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.275979 12116 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:49.277657 13407 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.277776 13458 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.271394 11551 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:49.215657 13418 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:49.216888 13412 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:49.211589 13609 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:15:49.281559 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:49.291818 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 85623us
I20260709 13:15:49.228962 13572 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:49.330075 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48619.1us
W20260709 13:15:49.332098 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 117754us
I20260709 13:15:49.229779 13594 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.280769 13573 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.278016 13438 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.285022 13412 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.340880 13412 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.284757 13564 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.286302 13631 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:49.282317 13581 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.216218 13623 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
W20260709 13:15:49.346829 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 137535us
I20260709 13:15:49.326927 13407 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.350273 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.352209 13550 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:49.337314 13570 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.355681 13391 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.367986 13606 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.364217 13541 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.362912 13407 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.375495 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.382294 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.401947 11475 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:49.403090 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:49.403823 11475 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:49.383463 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:49.378286 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.379045 13644 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:15:49.390045 13641 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.412390 13606 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.413009 11477 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:49.413089 13429 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:49.371929 13418 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.414060 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.414254 13418 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.415692 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:49.416478 13418 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.416819 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:49.417023 13418 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:49.417318 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:49.417528 13418 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.417951 13418 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.418141 11471 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:49.414727 13606 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:49.419134 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.380710 13575 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.380354 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:49.419821 11471 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:49.413638 13429 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.421054 13429 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:49.398720 13407 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.370313 13617 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.396852 13541 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.423094 12112 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:49.377082 13580 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.079s	user 0.005s	sys 0.000s
I20260709 13:15:49.425277 13399 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.412137 11300 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.433074 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.434089 11544 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:49.436517 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:49.437587 11544 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:49.287106 13595 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:15:49.361856 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:49.443449 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:49.446132 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.446323 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.446807 11544 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
W20260709 13:15:49.445226 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 82804.9us
I20260709 13:15:49.431079 11477 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:49.345005 13573 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.350625 13631 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.447521 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.480839 11471 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:49.523660 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:49.560006 11472 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.418064 13550 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:49.560508 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.502473 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.506665 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.561738 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:49.484455 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 144165us
W20260709 13:15:49.480665 11366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.650s (SENT) [suppressed 35 similar messages]
I20260709 13:15:49.449544 11400 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:49.606009 11401 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:49.448925 11300 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.610831 11472 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:15:49.441924 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.612295 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:49.616441 13522 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:49.625226 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.627681 11402 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:49.613938 13631 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.501036 13541 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.505589 13641 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.505942 13606 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.493472 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:49.498384 13624 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:49.665119 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:49.479874 13640 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:49.608702 13550 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:49.673941 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 193316us
I20260709 13:15:49.449347 13458 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.213s
I20260709 13:15:49.486907 13594 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
I20260709 13:15:49.445595 13399 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.483119 13476 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:49.685031 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 340014us
I20260709 13:15:49.636567 13617 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.736339 13418 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.737291 11399 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:49.691511 13574 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.738684 13418 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:49.739360 13476 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.739140 13574 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.741887 13574 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:49.679188 13624 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:49.723809 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.676110 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:49.730609 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:49.678069 13550 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.731714 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:49.672056 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:49.678383 13307 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.744486 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:49.745239 13506 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:49.749400 13598 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.301s	user 0.000s	sys 0.009s
I20260709 13:15:49.750288 13645 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.751410 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:49.675346 13640 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:49.761862 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:49.762712 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:49.824420 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.675652 13606 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:49.766695 13373 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.015s	sys 0.072s
I20260709 13:15:49.758013 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 364) took 1113 ms. Trace:
W20260709 13:15:49.779881 13562 consensus_meta.cc:287] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.002s
I20260709 13:15:49.688370 13455 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:15:49.846748 13449 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.852871 13399 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.855648 13606 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:49.859362 13572 consensus_meta.cc:287] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.002s
I20260709 13:15:49.687346 13535 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
W20260709 13:15:49.862170 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99506.9us
I20260709 13:15:49.879418 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:49.672540 13641 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:49.864775 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 80950us
I20260709 13:15:49.761140 13624 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.760951 13522 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:49.776597 13420 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:49.777951 13307 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.780097 13438 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.844372 13640 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.886384 11401 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:49.887395 11402 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:49.888803 11400 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:49.900933 11550 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:49.864542 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 102772us
I20260709 13:15:49.901738 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:49.890636 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.869271 11475 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:49.894629 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1067) took 1588 ms. Trace:
I20260709 13:15:49.907961 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:49.895247 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:49.893378 13636 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.889881 11544 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:49.902992 11546 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.870620 13650 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.913482 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.908887 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.907665 11395 rpcz_store.cc:276] 0709 13:15:48.306035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.306551 (+   516us) service_pool.cc:224] Handling call
0709 13:15:48.306937 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:48.350825 (+ 43888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.350856 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:48.351303 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:15:48.390557 (+ 39254us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:48.391584 (+  1027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.391647 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:49.748693 (+1357046us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:49.894369 (+145676us) raft_consensus.cc:1685] finished
0709 13:15:49.894390 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.894571 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45701,"raft.queue_time_us":45800,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":11718,"spinlock_wait_cycles":12219776,"thread_start_us":11436,"threads_started":1,"wal-append.queue_time_us":57469}
I20260709 13:15:49.898097 13420 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:49.915891 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.916945 11543 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:49.844605 13645 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:49.899878 13307 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.896996 13573 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:49.861714 12117 rpcz_store.cc:276] 0709 13:15:48.618096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.622730 (+  4634us) service_pool.cc:224] Handling call
0709 13:15:49.493305 (+870575us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:49.730971 (+237666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30075264}
I20260709 13:15:49.919340 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:49.882552 13572 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.936385 11546 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:49.893543 13564 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:15:49.911360 13541 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.943406 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:49.969758 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.916115 13636 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:49.911201 13551 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:49.931226 13407 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.956738 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:49.958364 11549 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:50.018234 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.008188 13636 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:50.047811 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.086480 13634 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:15:50.099448 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:15:50.120512 13660 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:15:50.127681 13641 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.018154 13551 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:49.973088 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.145212 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 438) took 1312 ms. Trace:
I20260709 13:15:50.146354 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.009349 13522 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.084913 13553 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.143743 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.183679 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.184140 11473 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:50.184485 11475 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:50.185287 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.185544 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.186321 11475 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:50.186609 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:49.934633 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:50.187641 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.182898 13535 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.182808 13564 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.189738 13535 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.190321 13564 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.192219 13564 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.196892 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.199681 11475 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:50.174197 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.204497 11471 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:49.958839 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:50.145677 12123 rpcz_store.cc:276] 0709 13:15:48.833240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.833377 (+   137us) service_pool.cc:224] Handling call
0709 13:15:50.144432 (+1311055us) spinlock_profiling.cc:243] Waited 976 us on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:15:50.144633 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.144844 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:50.144867 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:50.144915 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:50.144957 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.145018 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.145151 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2243968}
I20260709 13:15:50.207378 11546 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:15:50.223637 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.015s	sys 0.029s
W20260709 13:15:50.224409 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.015s	sys 0.029s
W20260709 13:15:50.242825 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 308251us
I20260709 13:15:50.243955 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:50.245376 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:50.276206 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 243) took 1006 ms. Trace:
I20260709 13:15:50.276702 11475 rpcz_store.cc:276] 0709 13:15:49.269749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.275242 (+1005493us) service_pool.cc:224] Handling call
0709 13:15:50.275605 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.275818 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:50.275854 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:50.275907 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:50.275954 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.276020 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.276140 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:50.192696 13535 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.280176 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:50.047803 13420 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:50.274392 13429 consensus_meta.cc:287] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.809s	user 0.010s	sys 0.001s
I20260709 13:15:50.307631 11475 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:50.193193 13573 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.280923 13591 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.318459 11546 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:50.273229 13564 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.297037 13650 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:50.230484 13553 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.313898 13429 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.343487 13572 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.329352 13650 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.321200 13573 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.324276 13467 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:15:50.374846 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1540) took 1712 ms. Trace:
I20260709 13:15:50.359738 13564 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:50.383019 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 102900us
I20260709 13:15:50.382377 11468 rpcz_store.cc:276] 0709 13:15:48.662709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.732220 (+ 69511us) service_pool.cc:224] Handling call
0709 13:15:48.757350 (+ 25130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.835714 (+ 78364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.835745 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:48.835787 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:48.835822 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.374518 (+1538696us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b5400063454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:50.374587 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.374768 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179767680}
I20260709 13:15:50.367142 13591 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.318241 13654 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:15:50.334647 13462 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:15:50.378811 13645 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.377285 13650 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:50.396765 13631 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.723s	user 0.009s	sys 0.000s
I20260709 13:15:50.419209 13677 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:50.423121 13680 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:50.424356 13573 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.425242 13669 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:15:50.423193 13550 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.004s
I20260709 13:15:50.426939 13550 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.429490 13550 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.414681 13396 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.425658 13553 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.422246 13631 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:50.444427 13412 consensus_meta.cc:287] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.103s	user 0.012s	sys 0.000s
I20260709 13:15:50.445025 13412 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.446403 11396 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:50.415130 13467 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.447793 13467 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.408152 13572 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:50.431499 13640 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.000s
I20260709 13:15:50.387115 13564 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:50.463536 13659 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:15:50.464588 13672 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:15:50.465786 13572 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:50.467744 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 128235us
W20260709 13:15:50.468508 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.477465 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:50.478183 13429 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:50.469379 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 84891.8us
I20260709 13:15:50.480232 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:50.480826 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:50.451506 13396 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.483650 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:50.484448 13462 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:50.484673 13654 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:50.486106 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1565) took 1299 ms. Trace:
I20260709 13:15:50.489616 11549 rpcz_store.cc:276] 0709 13:15:49.186559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.186938 (+   379us) service_pool.cc:224] Handling call
0709 13:15:49.187298 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.008812 (+821514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.485884 (+477072us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540033d1d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:50.486049 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772703104}
I20260709 13:15:50.490819 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.491051 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:50.491660 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:50.493017 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:50.493379 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.494990 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.495013 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:50.496018 11549 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:50.499334 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:50.500526 11542 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:50.501950 13396 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.501994 13641 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:50.502722 13641 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.503213 13641 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.507150 13664 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.497828 13553 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.508572 13553 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.510694 13553 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.498050 13535 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.513021 13572 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.515336 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.516768 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.517724 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.525534 11393 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:50.487507 13429 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:50.521461 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.525557 11471 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:50.529505 11402 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:50.530007 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.526441 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.531841 11474 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.535427 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.470793 13640 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.518507 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:50.536337 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:50.489167 13462 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.547869 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:50.545823 11474 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:50.549921 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:50.551103 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:50.556196 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:50.526257 13550 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.526752 13631 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.560613 13664 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.550010 13462 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:50.582213 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:50.585503 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.586229 11471 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.554890 13535 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.558632 13553 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.553776 13420 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:50.558908 13572 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.612704 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:50.612226 13553 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.619867 13553 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.616240 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1526) took 1368 ms. Trace:
I20260709 13:15:50.623787 11396 rpcz_store.cc:276] 0709 13:15:49.247793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.247939 (+   146us) service_pool.cc:224] Handling call
0709 13:15:49.248329 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.446900 (+1198571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.615878 (+168978us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400231254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:50.616152 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2754799872}
I20260709 13:15:50.574579 13664 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.560927 13550 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.563124 13429 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:15:50.604552 13631 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.604355 13541 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:50.623070 13675 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.222s	user 0.001s	sys 0.001s
W20260709 13:15:50.623188 13307 consensus_meta.cc:287] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.004s
I20260709 13:15:50.640460 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:50.613482 13420 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.618813 13572 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.652537 11471 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
W20260709 13:15:50.656287 13674 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:15:50.632001 13467 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.637025 13550 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.636930 13535 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:50.661530 13636 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.560s	user 0.004s	sys 0.009s
I20260709 13:15:50.634647 13664 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.636499 13412 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.638576 13631 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.776849 13631 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.763326 13564 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.644205 13420 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.660596 13429 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.658943 13553 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.671681 13572 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.664575 13541 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:50.779914 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:50.786792 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 202400us
I20260709 13:15:50.785467 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1547) took 1576 ms. Trace:
I20260709 13:15:50.660666 13682 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.135s
I20260709 13:15:50.667557 13307 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.790107 11477 rpcz_store.cc:276] 0709 13:15:49.209122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.417695 (+208573us) service_pool.cc:224] Handling call
0709 13:15:49.430671 (+ 12976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.430918 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:49.939261 (+508343us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.777032 (+837771us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.782252 (+  5220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:50.782326 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:50.782391 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.784990 (+  2599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.785191 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6741,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":58439,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":6801,"thread_start_us":834379,"threads_started":2,"wal-append.queue_time_us":447684}
I20260709 13:15:50.784322 13584 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:50.769415 13636 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.769332 13535 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:50.802350 13683 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:15:50.804198 13307 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.812816 13664 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.816221 13664 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:50.820768 13685 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:15:50.812811 13541 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.791631 13640 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.831492 13695 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:50.832261 13695 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:50.833488 13695 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.838248 13522 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.838951 13522 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.841723 13522 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:50.844257 13553 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:50.853071 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.856001 13572 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.864687 13641 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.872529 13694 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.873339 13694 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:50.787147 13429 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:50.875906 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 85817.1us
I20260709 13:15:50.795898 13584 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:50.890978 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:50.895944 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:50.897320 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:50.894850 11473 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.898656 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:50.868300 13572 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:50.894213 13664 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.911659 13595 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.921728 13681 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:50.922320 13681 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.905838 13591 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.911615 13686 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:50.894827 13564 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:15:50.914288 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:50.903532 13677 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:15:50.972994 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1539) took 2310 ms. Trace:
I20260709 13:15:50.933032 13572 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.968720 13686 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:50.983527 11394 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:50.971140 13650 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.000s
I20260709 13:15:50.977686 11470 rpcz_store.cc:276] 0709 13:15:48.662472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.731264 (+ 68792us) service_pool.cc:224] Handling call
0709 13:15:48.731625 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:48.783510 (+ 51885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.783540 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:48.783977 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:15:48.994298 (+210321us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:48.994353 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.968166 (+1973813us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b5400244354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:50.968233 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:50.968296 (+    63us) raft_consensus.cc:1685] finished
0709 13:15:50.968315 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.972912 (+  4597us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4913,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2192,"spinlock_wait_cycles":118302080,"thread_start_us":209962,"threads_started":1,"wal-append.queue_time_us":225605,"wal-append.run_cpu_time_us":1669,"wal-append.run_wall_time_us":1002265}
I20260709 13:15:50.945317 13664 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:50.996228 13641 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:50.997603 11473 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:50.906106 13573 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:15:50.967417 13640 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.019917 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.020785 11546 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:15:51.027961 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 129385us
I20260709 13:15:50.998526 13553 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.004673 13677 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.006564 13686 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.030572 11400 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:51.032992 11544 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:51.038089 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.038815 11550 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.036787 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.039434 11400 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:51.018111 13550 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.005529 13572 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.006953 13420 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.025161 13640 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.018807 13650 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.040051 13697 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.040534 13564 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:51.034848 13553 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.004082 13584 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.067420 13584 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.060179 13564 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.072531 13564 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.073009 13564 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.073568 13564 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.062383 13631 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.049054 13681 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.062850 13553 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.064366 13640 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.075906 13572 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.035751 13677 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:51.083227 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47344.1us
I20260709 13:15:51.058784 13699 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.088290 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:51.073572 11550 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:15:51.084220 13692 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:15:51.094028 13703 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:51.109671 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.094939 13553 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:51.095278 13462 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.115886 11394 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:51.096979 13595 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.098174 13535 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.079622 13681 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.101497 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.102391 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.119874 11400 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:51.103662 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1546) took 2435 ms. Trace:
I20260709 13:15:51.104058 11398 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.087028 13694 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.121555 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.116039 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.095772 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:51.118050 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.118430 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:51.113803 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:51.124464 11394 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:51.077208 13697 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.130029 13677 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.130604 13694 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.131781 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.133054 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.134156 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.120379 11469 rpcz_store.cc:276] 0709 13:15:48.668175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.985189 (+317014us) service_pool.cc:224] Handling call
0709 13:15:49.132156 (+146967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.132497 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.132544 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:49.132603 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:49.132648 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.132737 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.103428 (+1970691us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540042a4d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:51.103575 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336681728}
I20260709 13:15:51.125351 13467 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:51.125834 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.140249 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.141297 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.144438 13462 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.126190 13564 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.127147 13650 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.127444 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.128001 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.128722 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.128860 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.129837 13420 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.136221 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.136739 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.137131 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.178128 13564 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.178654 13564 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.179441 13686 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.109774 13703 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.180864 13703 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.138911 13681 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.128803 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.198506 13606 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:51.199141 13606 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:51.200232 13606 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.158980 13639 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.183987 13682 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.239326 13609 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:51.239815 13609 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.187412 13599 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.240690 13609 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.265697 13609 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.240661 13694 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:51.203400 13639 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.283043 13664 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.322432 13606 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:51.338894 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:51.339520 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 201658us
W20260709 13:15:51.146694 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50990.7us
I20260709 13:15:51.151117 13467 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:51.105007 13699 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.157444 13702 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.137923 13705 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.160156 13641 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.161903 13624 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.163108 13553 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:51.163439 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1558) took 2064 ms. Trace:
I20260709 13:15:51.373890 11548 rpcz_store.cc:276] 0709 13:15:49.099391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.099537 (+   146us) service_pool.cc:224] Handling call
0709 13:15:49.099947 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.231916 (+131969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.231965 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:49.232026 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:49.232072 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.163209 (+1931137us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54003ea3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:51.163256 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.163386 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302899584}
I20260709 13:15:51.163849 13550 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.118342 13631 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.167160 13535 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.342715 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1531) took 2093 ms. Trace:
I20260709 13:15:51.386240 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.349226 11469 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.287104 13669 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.350503 11476 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: [3.2-4.3]
I20260709 13:15:51.357025 13655 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.386447 11399 rpcz_store.cc:276] 0709 13:15:49.249225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.263117 (+ 13892us) service_pool.cc:224] Handling call
0709 13:15:49.263414 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.737773 (+474359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.342474 (+1604701us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540014a0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:51.342651 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089386880}
W20260709 13:15:51.351272 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.351823 11477 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.392313 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:51.393079 11477 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:51.351274 11470 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:51.356107 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:51.361182 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:51.363201 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1088) took 1419 ms. Trace:
I20260709 13:15:51.397497 11395 rpcz_store.cc:276] 0709 13:15:49.943641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.944170 (+   529us) service_pool.cc:224] Handling call
0709 13:15:49.944533 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.944753 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:49.974577 (+ 29824us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.155600 (+1181023us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b18003e1e80. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:15:51.362739 (+207139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:51.362803 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:51.362849 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.362961 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.363145 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28941,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":92,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":82579712,"thread_start_us":206618,"threads_started":1,"wal-append.queue_time_us":1387841}
I20260709 13:15:51.368254 13694 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.400588 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:51.400952 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:51.369395 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:51.302464 13703 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.283358 13599 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.382329 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.383955 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.264618 13682 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:15:51.385421 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.401875 11468 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:51.413519 12119 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:51.402559 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:51.358255 13686 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:51.426301 13710 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:15:51.429446 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1026) took 1518 ms. Trace:
I20260709 13:15:51.429998 11475 rpcz_store.cc:276] 0709 13:15:49.932469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.312841 (+380372us) service_pool.cc:224] Handling call
0709 13:15:50.313172 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:50.363703 (+ 50531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:50.363734 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:50.364266 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:15:51.175918 (+811652us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b18003d3c00. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:51.335328 (+159410us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:51.335386 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.335487 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.429069 (+ 93582us) raft_consensus.cc:1685] finished
0709 13:15:51.429099 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.429365 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8494,"raft.queue_time_us":47943,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3388,"spinlock_wait_cycles":17030528,"thread_start_us":200706,"threads_started":2,"wal-append.queue_time_us":1021210}
I20260709 13:15:51.431769 11547 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:51.406304 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45232.6us
I20260709 13:15:51.409289 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.409983 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.440510 13550 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.441045 13550 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.381292 13718 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.388141 13553 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:51.443885 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.444679 11543 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:51.444438 13553 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.443262 13718 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.447108 13681 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.448388 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.382083 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.448992 11545 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:51.420882 11548 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:51.424003 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.454094 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.455248 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:51.456650 11468 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.368970 13702 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.456709 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.460211 11548 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.450235 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:51.468282 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65840us
I20260709 13:15:51.392409 13581 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:51.476364 11545 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.399425 13564 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.373903 13641 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.420608 13631 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:51.422668 13573 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.412321 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.473766 11476 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.468371 11547 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:51.400161 13535 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.405959 13669 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.491012 11545 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.412585 13655 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.451299 13664 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.508857 11542 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.498162 11468 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.478760 13681 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:51.499459 13718 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:51.515349 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 176512us
I20260709 13:15:51.500941 13682 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.516144 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:51.501946 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.503508 11544 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.505318 13702 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.465756 13467 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.507447 11545 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.511312 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.373447 13705 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.418804 13694 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.483017 13581 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.516091 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.480437 13550 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.491513 13590 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:51.522338 12123 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:51.494632 13609 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.527475 11547 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:51.527493 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:51.507541 13631 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:51.477634 13712 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:51.495954 13573 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.532366 13458 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.516412 13664 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.538142 13590 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.543108 13609 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.553488 13703 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.558746 13699 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.558924 13680 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:51.559438 13680 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:51.558653 13718 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.562707 13669 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.562857 13705 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.563985 13667 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.564049 13697 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.564607 13667 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:51.564771 13664 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.558786 13694 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.563999 13712 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:51.566401 13712 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:51.596439 11366 leader_election.cc:336] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.650s (SENT)
I20260709 13:15:51.597739 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:51.534829 11542 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:15:51.607419 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79788.5us
W20260709 13:15:51.621417 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 175761us
I20260709 13:15:51.622150 13335 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.624014 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.625511 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:51.552647 13590 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.630381 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.532148 13522 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:51.562846 13655 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.640695 13697 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:51.641225 13697 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.641574 13697 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:51.545344 13631 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.648032 13639 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.653223 13609 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:51.664829 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53805.7us
I20260709 13:15:51.667445 11550 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:51.668776 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 136614us
I20260709 13:15:51.670701 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:51.671110 13522 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:51.671684 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:51.680119 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47490us
I20260709 13:15:51.558037 13636 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.687665 13599 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:51.679546 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 83372.6us
I20260709 13:15:51.696785 13641 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.698539 13551 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.701826 13507 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.702241 13159 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:51.703615 13159 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:51.704138 13159 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.702674 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:51.711160 13723 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.713348 13535 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.714251 13522 consensus_queue.cc:1243] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:51.716293 13581 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.723855 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:51.736598 11545 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.736675 11475 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.560292 13680 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.741187 13680 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:51.702850 13507 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.745651 13507 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.704512 13667 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.747634 13667 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.749611 11550 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:51.759150 13551 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.759799 13307 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.754961 11543 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.770789 13573 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.771692 13641 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.771553 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:51.769006 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 44534.4us
I20260709 13:15:51.707819 13159 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.731966 13609 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:15:51.759829 13551 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.788380 11545 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:51.785620 13335 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:15:51.772675 13641 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.802474 13636 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.808151 13644 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:51.795377 13535 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:51.810949 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 140551us
I20260709 13:15:51.812953 13159 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.813977 13458 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.816623 13458 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.789322 13573 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.819430 13573 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.820506 13551 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.821187 13429 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:51.821825 13429 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:51.822551 13429 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.826162 11476 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:15:51.827360 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 125093us
I20260709 13:15:51.819814 13498 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:15:51.827667 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56523.9us
I20260709 13:15:51.779625 13396 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.135s
I20260709 13:15:51.829764 13636 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.835597 13636 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:51.836050 13668 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.438s	user 0.005s	sys 0.001s
I20260709 13:15:51.836688 13718 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.837103 13718 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:51.821846 13573 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.828840 13498 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.838820 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:51.841698 13718 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.847640 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.848093 11544 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:51.848202 13641 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.848546 13718 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.841662 13458 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.850953 13697 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.852062 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.853318 12115 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:51.853966 13429 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.854482 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.856133 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.857029 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:51.858439 13606 consensus_meta.cc:287] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.006s
I20260709 13:15:51.858683 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.829752 13644 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.866974 13644 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.867645 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:51.868506 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 71365.4us
I20260709 13:15:51.868808 13617 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.873641 13498 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.874552 13617 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:51.875293 13498 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.875813 13617 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.877250 13617 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.870699 13335 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:51.871425 13571 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:51.880404 13571 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:51.880836 13571 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.881820 13571 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.872262 13207 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:51.883404 13207 consensus_queue.cc:1243] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:15:51.872200 13307 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:51.884603 13307 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:51.884950 13307 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.886093 11546 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:51.885926 13307 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.869935 13589 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:15:51.887766 13589 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:51.888178 13589 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.876418 13712 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.873651 13455 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:51.891079 13455 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.891439 13455 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.891746 13697 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:51.892377 13455 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.895350 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.895777 13571 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.896319 12117 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:51.898366 13573 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.903560 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.903386 13573 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.903803 13307 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.904600 11543 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:15:51.905746 13573 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.906239 11471 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.903399 13551 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:51.906973 13551 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.909614 13551 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:51.906524 13712 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:51.915418 13724 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 13:15:51.915755 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71362us
W20260709 13:15:51.917042 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 71139.8us
W20260709 13:15:51.916435 13726 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:15:51.919672 13507 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:51.920122 13507 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.921211 13507 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.890142 13589 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.924603 13589 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:51.927285 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.928180 11400 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:51.929562 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:51.929587 13631 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:51.930471 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.922772 13507 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.931432 13429 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:51.897410 13644 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.944252 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:51.951116 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:51.952798 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:51.954064 13641 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.954492 13641 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:51.955138 11395 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.955524 13641 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.956975 13641 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.957151 11397 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.961123 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:51.962141 11300 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 242) took 20887 ms (client timeout 15000 ms). Trace:
I20260709 13:15:51.963822 11550 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:51.964761 11469 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.967052 13455 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.947466 13631 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.980571 13396 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:51.981769 13396 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.981810 13590 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:51.980616 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:51.982288 13396 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.986263 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:51.986632 11476 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.987057 13576 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:51.987416 13609 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:51.987756 11402 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:51.987825 13609 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:51.988306 13644 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.989089 13609 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.971987 13697 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.987088 12116 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:51.990602 13396 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:51.987655 13576 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.001168 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.003710 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.006866 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:52.007437 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.007941 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:52.008244 13576 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.009287 13396 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.009801 13667 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:52.010335 13667 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.010712 13667 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.011101 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.011893 12116 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:52.013000 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.012898 13476 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:52.014088 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:52.014194 13335 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:52.014688 13335 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:52.009742 13717 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:52.015059 13407 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:52.015165 11545 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:52.014232 13476 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:52.015959 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.016731 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:52.019372 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89968.7us
I20260709 13:15:52.024653 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.022676 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.027796 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.028983 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:52.029794 13697 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.020759 11394 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:52.065189 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.066457 13697 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:52.067387 13722 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:52.031308 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.068954 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.069393 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.070020 11476 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:52.070129 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.032799 13631 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.033238 12117 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:52.071290 13631 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.071774 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.072899 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.073474 13631 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.074010 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.074106 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.033643 13335 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.075433 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.075910 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.076224 11551 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:52.076953 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.077247 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:52.078297 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64335us
I20260709 13:15:52.035178 13407 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:52.079176 13407 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.079169 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.079547 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.080365 11393 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.070710 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.063776 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:52.049244 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.049968 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.051640 11544 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:52.053431 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.056095 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.055773 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.090102 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:52.097827 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.057179 13476 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:52.063179 13159 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.109684 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.035089 13717 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:52.103643 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.111799 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.108110 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:52.112337 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.075933 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.115545 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.116568 11474 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:52.095415 13571 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:52.095703 13644 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.091308 12113 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:52.125994 13590 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:52.127651 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.131372 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.131924 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.132714 11546 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:52.104826 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.134135 12118 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:52.134070 13712 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:52.135003 13712 consensus_queue.cc:1243] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:52.134747 13476 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.136153 13522 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:52.136556 13476 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.136626 13725 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:52.137851 13159 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.136641 13522 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:52.140134 13159 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:52.139729 13522 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.141261 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.142127 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.142714 13462 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:52.143150 13462 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:52.143898 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.144059 13462 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.144546 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:52.145430 13462 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.145598 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.141422 13522 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:51.962635 11300 rpcz_store.cc:269] 0709 13:15:31.074536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.074779 (+   243us) service_pool.cc:224] Handling call
0709 13:15:51.961897 (+20887118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:50.024732 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:50.120474 (+ 95742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:50.120510 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:15:50.120527 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:50.120916 (+   389us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:50.121001 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:50.121027 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:50.121351 (+   324us) tablet.cc:701] Row locks acquired
0709 13:15:50.121375 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:15:50.121519 (+   144us) write_op.cc:270] Start()
0709 13:15:50.121653 (+   134us) write_op.cc:276] Timestamp: P: 1783602950121479 usec, L: 0
0709 13:15:50.121686 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:15:50.126397 (+  4711us) log.cc:844] Serialized 911 byte log entry
0709 13:15:50.761868 (+635471us) op_driver.cc:464] REPLICATION: finished
0709 13:15:51.029861 (+267993us) write_op.cc:301] APPLY: starting
0709 13:15:51.033449 (+  3588us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:51.041311 (+  7862us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:51.048694 (+  7383us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:51.108164 (+ 59470us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:51.108392 (+   228us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:15:51.108454 (+    62us) write_op.cc:312] APPLY: finished
0709 13:15:51.110722 (+  2268us) log.cc:844] Serialized 29 byte log entry
0709 13:15:51.121799 (+ 11077us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:51.138019 (+ 16220us) write_op.cc:454] Released schema lock
0709 13:15:51.139766 (+  1747us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:51.164347 (+ 24581us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:51.164401 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11194,"reported_tablets":10,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":254143,"apply.run_cpu_time_us":2920,"apply.run_wall_time_us":139842,"num_ops":2,"prepare.queue_time_us":344216,"prepare.run_cpu_time_us":35142,"prepare.run_wall_time_us":387922,"raft.queue_time_us":182971,"raft.run_cpu_time_us":79644,"raft.run_wall_time_us":279019,"replication_time_us":517594,"thread_start_us":540715,"threads_started":4,"wal-append.queue_time_us":294668,"wal-append.run_cpu_time_us":32157,"wal-append.run_wall_time_us":1707791}]]}
W20260709 13:15:52.149616 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51888.5us
I20260709 13:15:52.150895 13576 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.153506 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.154407 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:52.154871 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.155349 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.156037 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.156522 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.157265 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:52.157562 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.157683 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.157889 11468 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:52.158834 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.159430 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.160727 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.161356 11470 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.161759 11545 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:52.163146 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:52.164309 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 66793.3us
I20260709 13:15:52.153013 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.144718 11548 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:52.054842 13667 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.165943 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:52.175011 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.176205 11474 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:52.180058 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.181093 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.182811 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.181334 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.183211 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.186120 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:52.198220 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47849.7us
W20260709 13:15:52.199019 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 91007.9us
I20260709 13:15:52.114199 13717 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.219014 11474 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:52.218451 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 90398.2us
I20260709 13:15:52.224994 11469 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.226388 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.228332 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.182904 13722 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:15:52.238011 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.219360 12115 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:15:52.243206 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 92300.5us
W20260709 13:15:52.248013 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 67204.1us
I20260709 13:15:52.197012 13609 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.232069 12116 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:52.252086 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.196772 13335 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.255738 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.264286 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.265138 11546 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:52.272096 11367 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.274168 11397 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:52.263839 11471 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.279261 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.282702 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.285260 13554 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:52.285388 11299 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.287377 11299 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.289193 13667 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.288995 11299 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.289937 13055 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.291114 13574 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:52.291652 13574 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:52.292104 13574 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.294823 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.292622 11299 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.297943 11299 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 3 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.304865 13335 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:52.305441 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.306190 11299 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 3 to 6, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 6 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:52.313704 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 45313us
W20260709 13:15:52.285449 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 61386.6us
I20260709 13:15:52.316355 12112 raft_consensus.cc:2410] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:15:52.317018 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:52.318038 13697 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:52.318781 13697 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.318185 13631 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:52.322118 13631 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:52.322626 13631 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.260732 13725 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:52.328121 11393 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
W20260709 13:15:52.329061 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57098.3us
I20260709 13:15:52.258201 13641 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.335186 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.332003 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.339114 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.336148 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:52.343504 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 83854.1us
I20260709 13:15:52.345783 11543 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:52.347420 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:52.347777 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 161334us
I20260709 13:15:52.326278 13697 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.348487 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.333858 13590 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:52.364595 11546 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:52.358407 11477 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.359156 11299 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:52.370465 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 43478.6us
I20260709 13:15:52.373905 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.348340 13498 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.381356 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.384758 13444 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:52.389683 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.391826 13606 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:52.393060 13731 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:15:52.347220 13641 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.377617 11299 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.366182 13722 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:52.411321 13307 consensus_meta.cc:287] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.004s
I20260709 13:15:52.421221 11394 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:52.408701 11395 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:52.410181 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.373857 13590 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:52.426764 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.425994 11299 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.430248 11471 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:52.432039 11544 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:52.427553 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:52.437059 11470 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:52.426760 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:52.449659 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.450531 11469 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.453418 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.453751 13590 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:52.460057 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 91483.1us
I20260709 13:15:52.463832 11395 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:52.464125 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.462623 11299 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 3 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:52.475965 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 76077.9us
W20260709 13:15:52.479554 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54976.2us
I20260709 13:15:52.480396 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.480818 13571 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:52.481352 13571 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:52.403152 13574 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.510370 11474 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:52.511905 11476 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.470817 13644 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:52.491611 13498 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:52.521471 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.525568 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.498643 13563 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.535066 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.535753 11395 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:15:52.522881 13732 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:15:52.543547 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 176840us
I20260709 13:15:52.474476 13581 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:15:52.545823 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.545994 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.547960 13641 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.551098 13655 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:15:52.551944 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.552886 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:52.553412 13737 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:15:52.482312 13571 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.554977 11399 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:52.562333 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:52.563699 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.564823 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.556955 11299 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.567618 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.569214 13641 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:52.571650 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.573421 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.578188 11397 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:15:52.576938 11299 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 4, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.581404 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.582451 11395 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:52.568470 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.587875 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.587997 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.590636 11546 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:52.545357 13563 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.603719 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.608580 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.605801 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.606308 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 242) took 1248 ms. Trace:
I20260709 13:15:52.610602 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.610718 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:52.610538 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 229235us
I20260709 13:15:52.609828 11398 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.614622 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.619324 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.620961 11393 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:52.621045 11394 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:52.621771 11395 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.596375 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:52.627658 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.632524 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.632704 11400 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:15:52.633080 11400 pending_rounds.cc:85] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 2
I20260709 13:15:52.633184 11398 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:52.633334 11400 pending_rounds.cc:107] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:15:52.633648 11400 raft_consensus.cc:2889] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:52.633857 13563 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.634181 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.634575 11397 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:52.635056 11400 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:52.635290 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.635915 11402 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:52.636283 11400 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:52.636888 11402 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:52.611112 11401 rpcz_store.cc:276] 0709 13:15:51.321905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.920826 (+598921us) service_pool.cc:224] Handling call
0709 13:15:51.921457 (+   631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.921954 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:51.975730 (+ 53776us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.238096 (+262366us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:52.263121 (+ 25025us) log.cc:844] Serialized 26 byte log entry
0709 13:15:52.263266 (+   145us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:52.263456 (+   190us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.263688 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.263760 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:52.431323 (+167563us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b2400137078. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:15:52.566332 (+135009us) raft_consensus.cc:1685] finished
0709 13:15:52.566353 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.570287 (+  3934us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7616,"prepare.queue_time_us":286411,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":10948,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":9038,"spinlock_wait_cycles":22556032,"thread_start_us":268058,"threads_started":2,"wal-append.queue_time_us":309087}
I20260709 13:15:52.642397 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.643786 13055 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.644428 13055 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.591830 13641 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.611351 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:52.627938 11299 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 2 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.660611 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.620132 13697 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:52.695513 13055 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.705039 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.696081 11394 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:52.696456 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.705945 11398 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:15:52.708561 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 220631us
W20260709 13:15:52.715447 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 162747us
I20260709 13:15:52.718443 11395 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:15:52.712994 13055 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:52.712987 11299 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.724619 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.738649 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.744735 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.723894 13574 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.709159 13697 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.731257 13498 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:52.753058 11299 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 5, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 5 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:15:52.759133 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 55639.4us
I20260709 13:15:52.737280 13644 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:52.761693 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.762235 13741 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:15:52.763322 13697 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.760686 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:52.792285 11542 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:52.764487 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:52.767165 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.767628 13641 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:52.767670 13725 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.773331 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.795547 13725 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.796309 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.775928 11299 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.777257 13631 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.799367 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:52.799320 11299 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 4, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.800104 12116 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:52.800933 11299 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.803270 13522 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:52.803426 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.803730 13522 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:52.804149 13522 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.804334 13742 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:52.806313 13617 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:52.806671 13617 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:52.803297 13722 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:52.807487 13722 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:52.807919 13722 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.807574 13617 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.808852 13617 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:52.808979 13407 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.003s
I20260709 13:15:52.779268 13753 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:52.779966 13745 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:15:52.780588 13743 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:15:52.822288 13754 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:52.780927 13717 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 13:15:52.786727 13680 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:52.824465 13680 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:52.822930 13754 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:52.825572 13680 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.825956 13754 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.826764 13680 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.815413 13631 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:52.795048 13756 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:52.828568 13756 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:52.828950 13756 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.813666 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.793473 11475 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:52.831759 13507 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:52.832015 11366 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.817777 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.833483 13458 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:52.833709 13476 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:52.834460 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:52.835392 13667 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:15:52.819994 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54985.1us
I20260709 13:15:52.820796 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.824451 13717 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.818507 13753 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.802438 11299 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.837109 11397 raft_consensus.cc:2376] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6. Re-sending same reply.
I20260709 13:15:52.849215 11299 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.850968 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.853540 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.835794 13667 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:52.835991 13476 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:52.794278 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:52.809460 13407 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.861070 13667 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.862075 13407 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:52.862404 13667 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.863035 13476 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.863474 13476 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.858166 11299 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 2 to 4, term changed from 1 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to <none>, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.864415 11299 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 3, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.838681 13697 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:52.859205 13631 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:52.859859 13606 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:52.868173 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:52.866000 11299 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 6, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 6 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.869112 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:52.867654 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.880263 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:52.872470 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.886897 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:52.911633 11299 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 1 to 5, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:52.850761 13753 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.912369 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:52.919844 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 165830us
I20260709 13:15:52.870474 13606 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.857795 13746 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:52.929273 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61686.8us
I20260709 13:15:52.888571 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:52.935745 12119 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:52.917575 13697 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:52.883428 13055 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:15:52.926733 13746 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:52.915987 13680 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:52.870973 13641 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:15:52.925691 13606 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:52.949329 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 80272.9us
I20260709 13:15:52.949976 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:52.934345 13589 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:52.885766 13725 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.859234 13644 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.947516 13712 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:52.948000 13746 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.956357 13407 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:52.959645 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:15:52.978307 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.979744 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77331.6us
I20260709 13:15:52.981135 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.960287 13755 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:52.952139 13697 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:52.979795 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:52.989522 11550 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:52.992882 11397 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:52.989543 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:52.994840 11399 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:52.954643 13758 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:52.951566 13680 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:52.999617 11543 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:53.005030 11551 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:52.951998 13522 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:52.986253 13712 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:52.997870 13571 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:15:53.002292 13758 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.038774 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:53.081049 12116 raft_consensus.cc:2368] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:15:52.984014 13429 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:53.116783 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 78083.3us
I20260709 13:15:53.111445 13429 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.111639 13755 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:53.109268 13758 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.114349 13754 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.175352 13754 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:53.169153 13760 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:53.177667 13754 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.178210 13754 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.178649 13754 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:53.180008 13476 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.181144 13717 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.182564 13476 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.161859 13757 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.104343 13462 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:15:53.169243 13680 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.186954 13746 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.190217 13307 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:53.221292 13307 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:53.222136 13307 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.129122 13697 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.195222 13722 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.236837 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:53.160892 13712 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.167313 13429 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.213928 11515 leader_election.cc:400] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:15:53.208647 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:53.197006 13759 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:15:53.211035 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 105339us
I20260709 13:15:53.249469 11515 leader_election.cc:403] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:15:53.167809 13755 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:53.257746 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 90652.1us
I20260709 13:15:53.258438 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:53.256367 13725 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.257499 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.267043 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:53.296756 11397 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:53.273329 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:53.298301 11476 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.275370 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:53.248329 13746 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.304774 11394 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:53.304967 13773 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.277547 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.305536 13755 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.275653 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:53.278822 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.301798 11472 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:53.302073 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:53.308071 11398 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.274400 12119 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:53.277144 13772 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:53.313355 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54990.6us
I20260709 13:15:53.314626 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:53.317602 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.252547 13757 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.320958 13757 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.321880 13522 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.321787 13767 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:53.291558 12112 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:53.260200 13753 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.263370 13742 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:15:53.351222 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.352712 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:53.353582 13753 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:53.353458 13574 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.350416 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:53.330668 11398 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:53.262864 13391 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:53.360059 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.361208 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:53.306639 13755 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.272454 13722 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.266486 13718 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:15:53.370141 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:53.311757 13644 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:53.350641 13055 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:53.369853 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:53.377233 12119 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:53.350397 13763 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:53.380180 13055 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:53.380738 11549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:53.381136 13768 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:53.382541 13631 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:53.350440 13680 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:15:53.384542 12083 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.384924 11549 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:53.366145 13391 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:53.366838 13773 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:53.368212 13641 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:53.274163 13606 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.392800 11549 raft_consensus.cc:614] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:15:53.408314 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78391.8us
W20260709 13:15:53.411484 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.384610 13763 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:53.287374 13754 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.386642 13055 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:53.413892 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:53.429487 11550 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:53.401059 13641 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:53.402477 13631 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:53.404719 13768 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:53.411818 13722 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:53.435676 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:53.442045 11546 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:53.435988 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50900us
I20260709 13:15:53.382566 13429 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.405198 13773 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:53.422649 13763 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:53.447937 11551 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:53.448568 13781 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:53.456313 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 71834.6us
I20260709 13:15:53.476444 11397 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:53.459085 13655 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:53.416527 13644 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:53.483538 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44681.2us
I20260709 13:15:53.487217 12119 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:53.433240 13581 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:53.488842 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.505195 11546 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:53.506343 11544 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:53.485908 13756 consensus_meta.cc:287] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.002s
I20260709 13:15:53.463187 13750 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.462917 13307 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.513099 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:53.519099 13772 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:15:53.471666 13773 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.501572 13581 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:53.522876 13712 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:53.599270 11476 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:53.467545 13631 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:53.615140 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 221431us
I20260709 13:15:53.424783 13758 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:53.547164 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.599537 13777 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:53.616613 13764 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:53.618019 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 112036us
I20260709 13:15:53.580766 13655 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:53.622705 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 109841us
I20260709 13:15:53.619819 13396 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.470970 13391 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.657989 11546 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:53.643396 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.574348 12119 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:53.510056 13756 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.634259 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:53.633739 13522 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.654011 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:53.629616 13335 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:53.653944 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:53.656498 13786 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:15:53.650452 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 132337us
I20260709 13:15:53.488505 13781 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.642261 13581 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.575798 13750 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.642829 13777 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:53.667902 13783 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:15:53.650133 13772 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:53.671108 13781 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.661471 13507 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:15:53.671734 13644 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.674114 13335 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.675180 11474 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:53.613095 13641 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.677206 11399 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:53.645975 13768 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.664353 13778 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:53.665989 13753 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.696352 11476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f946b7973fe482681135635493ebd48"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:55518
I20260709 13:15:53.677963 13522 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:53.661495 13396 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.702232 13335 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:53.690652 13307 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:53.657063 13655 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.712942 11543 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:53.682705 13589 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:15:53.724007 13396 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.697850 13764 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:53.713933 13307 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.725346 13764 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:53.726418 13764 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.727779 13764 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:53.728255 13750 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:53.719734 13775 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:15:53.721599 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:53.704305 12112 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.684597 13772 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.706636 13753 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:53.708389 13778 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:53.729149 13641 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:53.747833 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82586.9us
I20260709 13:15:53.746868 13429 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.757174 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.751421 13718 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.754218 13655 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.753861 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:53.766502 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:53.773473 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:53.766302 13718 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:53.800648 13790 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:15:53.803069 12119 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:53.801906 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.773161 13754 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:53.784293 13778 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.767503 13791 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.762969 13522 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.799899 13476 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.811304 13476 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.812489 12112 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:53.817286 13756 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:53.819339 13476 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.802356 13718 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.767526 13307 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:53.806282 13753 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.823845 13756 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.824893 13476 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:53.765887 13758 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:53.802573 13788 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.225s	user 0.005s	sys 0.000s
I20260709 13:15:53.803433 13429 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.833727 13391 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.803040 12115 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:53.805395 13754 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:53.838735 13754 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.839125 13429 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.840438 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 280) took 1029 ms. Trace:
I20260709 13:15:53.840839 11474 rpcz_store.cc:276] 0709 13:15:52.810849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.813821 (+  2972us) service_pool.cc:224] Handling call
0709 13:15:52.814183 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.689020 (+874837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.840202 (+151182us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400062a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:53.840379 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979668352}
I20260709 13:15:53.842145 13429 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.833276 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.839196 13055 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.850512 11471 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:15:53.851692 11471 pending_rounds.cc:85] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 2
I20260709 13:15:53.852020 11471 pending_rounds.cc:107] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:15:53.852335 11471 raft_consensus.cc:2889] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:53.822474 13581 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.861092 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.867162 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:53.869156 13800 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:53.881057 13772 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.865693 13055 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.885032 11470 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:15:53.887509 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.891135 13772 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:53.892647 13772 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.867348 13680 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.898408 13606 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.899147 13775 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.899533 13763 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.899658 13775 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.901787 13763 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.902386 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:53.909701 13768 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:53.910162 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:53.902866 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.913201 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.913582 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.913748 13680 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:53.902930 13581 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.903399 12119 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:53.857203 13755 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:53.898401 13773 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:53.915099 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.894739 13772 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:53.958076 13722 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:15:53.968462 11476 raft_consensus.cc:606] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:53.969014 11476 raft_consensus.cc:614] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:53.967396 13799 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.970590 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:53.972692 12119 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:53.975981 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.965730 13768 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:53.978677 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.980628 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.983659 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.978225 13755 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.981530 13757 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:15:53.988415 13773 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.988723 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.984186 11473 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:53.989766 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.990949 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:53.994527 11551 raft_consensus.cc:2410] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:53.974702 13712 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.960079 13754 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:53.997713 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:53.998824 11475 raft_consensus.cc:2410] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:53.977967 13778 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:54.000593 13778 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.004088 13778 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:53.976939 13775 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:54.010150 13712 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.012359 13712 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.013665 13462 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:54.014223 13462 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:54.015194 13462 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.020077 13462 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:54.035424 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.036324 11399 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.040427 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.041163 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.041745 11469 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:54.046228 11474 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:54.047056 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.047600 11400 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.050559 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.050025 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:54.053725 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45477us
I20260709 13:15:54.054863 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.055820 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.056545 11545 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:54.057914 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:54.061039 11399 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:54.053824 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.063333 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.063405 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:54.067417 11401 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.067266 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.067777 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.040380 13576 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.071087 11365 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.062578 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.064297 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.062475 13609 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:54.069835 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.057797 13476 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.073336 11396 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:54.066193 13507 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.111290 13507 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.070366 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.071969 11402 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:54.072748 13335 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.113128 13335 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.113343 13507 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.113576 13335 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.081806 13396 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.115594 11398 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:54.115500 13777 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.083562 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.116878 13777 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.117507 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.118634 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:54.119236 13773 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:54.069418 13722 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:54.119624 13773 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:54.119829 13722 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.120087 13773 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.120230 13722 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.084203 13754 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.069274 13768 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.121500 13754 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.084379 11394 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:54.072618 13498 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:54.086175 13798 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.383s	user 0.000s	sys 0.003s
I20260709 13:15:54.074967 13576 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.124512 11397 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:54.088195 11401 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:15:54.088651 13809 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:15:54.090607 13055 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.097533 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:54.099738 12117 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:54.109361 13476 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.133396 11395 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:54.128892 11400 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:54.135087 13718 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:15:54.135385 13476 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.136206 13775 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:54.136682 13775 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:54.144096 13055 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.146373 13055 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:54.146976 13768 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:54.147706 13768 consensus_queue.cc:1243] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:54.149703 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:54.150015 12118 pending_rounds.cc:85] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 6
I20260709 13:15:54.150316 12118 pending_rounds.cc:107] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:15:54.150537 12118 raft_consensus.cc:2889] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:54.137564 13775 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.157021 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.165849 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.167196 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.167972 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.169483 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.169559 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:54.138741 13799 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:54.170369 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.196327 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.163390 13563 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.179407 11549 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:54.182407 11397 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:54.181375 11551 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:54.181876 11542 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:54.171787 13767 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:54.168357 13725 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.185104 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.200570 13563 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.200443 13725 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.201050 13563 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.202677 13725 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:54.187808 13576 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.188382 13159 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.209080 13773 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.209259 13159 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.211513 13159 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.189666 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.190271 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.190457 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.192546 13763 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.211730 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:54.171465 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 100492us
I20260709 13:15:54.229367 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.230538 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.238770 11545 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:54.208573 13563 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.248064 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.244086 11544 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:15:54.245379 13763 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.179075 11543 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:54.198247 11542 raft_consensus.cc:614] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:54.203742 13767 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:54.187232 12112 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:54.258955 11399 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:54.259889 11543 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:54.204012 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.180816 13712 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.207093 13335 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.196713 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:54.263332 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66957.9us
I20260709 13:15:54.251268 11547 raft_consensus.cc:2368] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:15:54.251377 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.264792 11549 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:54.266480 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.266759 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.267222 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.267791 11471 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:54.267128 11476 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:54.254813 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:54.255596 11400 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:54.256729 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.269671 11397 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:54.212463 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.226414 11551 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:15:54.250806 11550 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:54.193817 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.235627 13754 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.268321 11469 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:54.249435 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.272619 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:54.273252 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 192953us
I20260709 13:15:54.273612 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.226624 13799 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:54.280376 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.274530 11398 raft_consensus.cc:2368] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:15:54.275055 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.284216 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.286054 11402 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:54.256232 13763 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.265776 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.267514 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.278087 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.286226 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.284284 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.284451 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.276544 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:54.292723 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 98996.9us
I20260709 13:15:54.293432 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.296914 11551 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:54.265089 13773 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:54.300563 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45838.5us
I20260709 13:15:54.305112 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.300904 11398 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:54.301476 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.297183 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.298207 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.302595 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.262501 13772 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:15:54.284024 13799 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:54.271999 13768 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:54.306797 11477 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:54.307384 11551 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:54.313494 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:54.317912 11394 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:54.279605 13757 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.326179 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.290104 13581 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.306340 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.324124 13644 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:54.322690 13758 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:54.298543 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:54.324831 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.343398 11549 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:54.333220 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.301347 13774 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.344534 13774 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.345027 13774 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.301486 13590 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:54.299945 13429 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:54.272085 13712 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.330742 11402 raft_consensus.cc:2368] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:15:54.280092 13391 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:54.337332 13498 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:54.349097 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.365981 11470 raft_consensus.cc:2376] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5. Re-sending same reply.
I20260709 13:15:54.367002 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.349781 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.320483 13802 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:54.281023 13476 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:15:54.342499 13718 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:54.368072 11396 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:54.379122 13522 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:54.342034 13757 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.370210 13802 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:54.389626 13802 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.390580 13757 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.355352 13590 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:54.391448 13590 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:54.391932 13335 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.392737 13712 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.382898 11402 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:54.383131 13722 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.387714 13772 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.336624 13768 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.337091 13793 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:15:54.407209 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.388533 13055 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.341879 13606 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:54.355592 13644 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.398509 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:54.402594 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97887.7us
I20260709 13:15:54.355999 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:54.405427 11393 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:54.342186 13581 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.409425 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.408043 11402 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:54.430578 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.433837 11473 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:15:54.433357 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 136707us
W20260709 13:15:54.445075 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 118979us
W20260709 13:15:54.445040 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 138790us
I20260709 13:15:54.449816 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.451870 11399 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:54.455284 11399 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:54.456342 13835 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:54.456928 13835 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.457039 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.457351 13835 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:54.457501 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9e9c2603c0bd4d25b713649b3890c46b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:15:54.458715 13837 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:54.459311 13837 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.462134 11544 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:54.463287 13758 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:54.464331 13769 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:54.464896 13758 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.470155 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.422230 13055 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.476531 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.479945 11470 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:54.481896 13722 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.482503 13581 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.443390 13498 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:54.484285 13498 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.484375 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.484745 13498 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.484978 11543 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.488679 13563 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.488986 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.489315 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.489620 11474 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:54.490501 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.490676 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.490973 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.491465 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.491766 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.492225 11471 raft_consensus.cc:2410] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:54.488546 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.492694 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.478888 13055 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.494822 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.495419 11472 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:54.492038 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.496421 11477 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:54.496958 11474 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.499153 13590 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.500394 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:54.505357 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.505518 13590 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.506027 11469 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:54.508044 13775 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:54.508777 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.509382 11473 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:54.366636 13767 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.510890 11402 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:54.511684 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.510618 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.515529 11473 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:54.523696 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.524798 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:54.525066 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.525414 11471 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:54.534440 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 215826us
W20260709 13:15:54.534893 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54194.6us
W20260709 13:15:54.510360 13783 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:15:54.535567 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.564635 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.468003 13837 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:54.569208 11543 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:54.570784 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:54.570967 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.571676 13722 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:54.572161 13722 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:54.573261 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.576835 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.577350 11472 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:54.577775 11472 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:54.512444 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:54.584627 11551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930d965be7eb49448fa23456beb9925a"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:54.585510 11551 raft_consensus.cc:606] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:15:54.585678 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.585984 11551 raft_consensus.cc:614] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:54.586946 11550 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.592973 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.592233 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.598183 11474 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:54.605489 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.605922 11543 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:54.606391 11472 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.607343 11472 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.607766 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.608084 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.609805 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.598697 11546 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:54.603495 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.597534 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.604125 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:54.611229 11476 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.612178 11402 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:54.579571 13571 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.613729 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.613324 11395 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:54.616760 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.618877 11475 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:54.618892 11397 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:15:54.626466 11550 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:15:54.615043 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:54.588109 13429 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:54.573288 13722 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.622107 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.637694 11476 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:54.611918 13574 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.664374 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.630447 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.663655 11442 leader_election.cc:400] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:15:54.671326 11442 leader_election.cc:403] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.597501 13606 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:15:54.666998 11402 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:54.674897 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 58281.2us
I20260709 13:15:54.614256 13571 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.627753 13825 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.672552 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.665997 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.642910 13476 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.698597 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:54.674279 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 52243.4us
I20260709 13:15:54.628391 13609 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:54.675662 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.615684 13755 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:54.672932 13777 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.553s	user 0.012s	sys 0.007s
I20260709 13:15:54.627807 13581 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:15:54.639474 13839 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.666790 13833 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:54.665517 13837 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:54.724449 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.674032 13574 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.689060 13571 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.681455 13757 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:15:54.732777 13833 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.748435 13833 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:54.728513 13609 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.752198 13836 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.753031 13609 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.752724 13836 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.753835 13838 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.754815 13838 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.719388 13476 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.755893 13476 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.759462 13644 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.759955 13522 consensus_queue.cc:1243] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:54.760478 13498 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.762391 13498 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:54.763002 11401 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:54.763149 13718 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:54.763674 13718 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:54.763892 13429 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:54.766291 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.741266 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:54.738312 13837 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:54.738709 13574 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.729063 13773 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.736323 13576 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 13:15:54.746227 13821 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:15:54.746553 13819 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.747255 13827 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:54.725669 13756 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.770951 11442 leader_election.cc:400] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
W20260709 13:15:54.783188 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 160264us
I20260709 13:15:54.786435 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.788254 11550 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:54.790572 11550 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:54.690027 13835 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.799744 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:54.802918 11543 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:54.743180 13768 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:54.784897 13809 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.056s	user 0.001s	sys 0.003s
I20260709 13:15:54.778788 13644 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:54.831171 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.813707 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.830777 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.838917 11542 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.770093 13755 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.808670 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:54.868747 11547 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:54.836932 11471 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:54.842509 11545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:15:54.843218 13839 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.843794 13838 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:54.846856 13055 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.835573 13773 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:15:54.852174 13837 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.855247 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:54.887437 11401 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:54.836601 13793 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.862308 13476 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:54.864773 11472 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:54.850998 13836 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:54.870666 11545 raft_consensus.cc:606] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:54.769047 13791 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:15:54.857473 13802 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:54.880026 13799 consensus_meta.cc:287] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.005s
W20260709 13:15:54.891069 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46203.6us
I20260709 13:15:55.005800 11545 raft_consensus.cc:614] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:54.875183 13828 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:55.016263 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 339903us
I20260709 13:15:54.870081 13768 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 13:15:55.020366 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:54.882483 13498 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:55.022895 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:55.024539 11442 leader_election.cc:400] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:15:55.025025 11442 leader_election.cc:403] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:15:55.012549 13815 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:15:55.027982 11398 raft_consensus.cc:2368] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:55.004283 13756 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.030944 11549 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:55.037189 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:54.802229 13819 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.033717 13833 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.083851 13833 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.084671 13833 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:55.024228 13576 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:55.117498 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 238251us
W20260709 13:15:55.123252 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 102996us
I20260709 13:15:55.125218 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.118499 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:54.883380 13839 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.091905 13407 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:15:55.027321 13768 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:55.109786 13841 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.127s	user 0.000s	sys 0.002s
W20260709 13:15:55.032230 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 315031us
I20260709 13:15:55.024557 13799 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.133901 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:55.038230 13757 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
W20260709 13:15:55.127429 13763 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.814s	user 0.005s	sys 0.013s
I20260709 13:15:55.075991 13609 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:15:55.147573 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:55.087320 13581 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:15:55.118301 13576 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.077198 13756 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.031652 13644 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:15:55.159008 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.139802 13791 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:55.117138 13773 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:15:55.030515 13498 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:55.177188 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 52076.7us
I20260709 13:15:55.172122 11468 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:55.035465 13755 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:15:55.028292 13828 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.165678 13768 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:55.180044 11401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:55.163901 13589 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.162832 13763 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.203186 13763 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.168424 13791 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:55.219615 13791 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.220785 13791 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.140878 13844 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:15:55.177997 13845 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.225179 13839 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.229429 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:55.230077 12119 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:15:55.178265 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.197216 13819 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.197916 13498 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.198102 13828 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.198943 13768 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.199867 13802 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.192654 11401 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:15:55.200709 13589 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:55.207291 11548 raft_consensus.cc:2368] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:15:55.213792 11473 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:55.216339 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.244413 11475 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:55.226172 13777 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.192780 13835 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.201166 13849 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:55.200064 13856 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:15:55.194128 13774 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.000s
I20260709 13:15:55.168108 13576 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.235852 11547 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:55.234017 13851 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:55.275830 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97680.4us
I20260709 13:15:55.277498 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:55.282847 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 244168us
I20260709 13:15:55.284518 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:55.242904 13763 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.247573 13589 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.246436 13055 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.263350 13831 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.296687 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:55.275177 13851 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:55.315740 13858 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.316304 13858 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.318177 13522 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:55.323922 13571 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.002s
I20260709 13:15:55.324512 13571 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.327732 13799 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.328203 13576 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.328667 13839 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.329239 13839 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:55.329726 13839 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.332479 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.291594 13581 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:15:55.311038 13835 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.263757 13777 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.318713 13522 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.298462 13763 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.271754 13849 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.355432 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.382268 11547 raft_consensus.cc:2368] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
W20260709 13:15:55.351555 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.373054 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.274490 13774 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.384023 11550 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:55.360582 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:55.385022 11542 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.002s
W20260709 13:15:55.395988 13857 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:55.405510 11542 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
W20260709 13:15:55.387115 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54754.1us
I20260709 13:15:55.407824 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:55.418207 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:55.450464 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:55.451272 11545 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.417438 11551 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:55.344348 13858 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.374414 13571 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.339151 13799 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
W20260709 13:15:55.465929 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 58222.1us
I20260709 13:15:55.467010 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:55.469166 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.477433 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:55.381343 13429 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.416096 13758 raft_consensus.cc:2764] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:15:55.457158 13756 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.465391 13055 raft_consensus.cc:2764] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:55.475482 13755 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:55.540511 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63172.8us
I20260709 13:15:55.541640 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:55.484025 13718 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.003s
I20260709 13:15:55.481379 12117 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:55.481671 13845 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:55.481915 13844 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.483898 13396 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.558423 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.489527 13799 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.487789 13791 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.403164 13851 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.538341 13835 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.537232 13754 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.562824 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:55.536814 13756 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:55.492012 13722 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:55.662444 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 99723.6us
I20260709 13:15:55.664152 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.666288 11545 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:55.658106 13856 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:55.567633 13802 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.563294 13429 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:55.550446 13755 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.552093 13718 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.689148 11396 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:15:55.689627 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 329161us
I20260709 13:15:55.690095 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.660945 13799 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:55.696350 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 275) took 1524 ms. Trace:
I20260709 13:15:55.662317 13756 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:55.573799 13396 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.727149 11393 rpcz_store.cc:276] 0709 13:15:54.161957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.162078 (+   121us) service_pool.cc:224] Handling call
0709 13:15:55.685656 (+1523578us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:55.686143 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479876480}
I20260709 13:15:55.672299 13869 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.718818 11442 leader_election.cc:400] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
W20260709 13:15:55.718245 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 231770us
I20260709 13:15:55.669428 13845 raft_consensus.cc:2764] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:55.725183 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:55.737430 11368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.018s	sys 0.019s
W20260709 13:15:55.737798 11368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.018s	sys 0.020s
I20260709 13:15:55.738132 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.610281 13791 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.753605 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.754527 11393 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:55.730669 11475 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:55.736603 11442 leader_election.cc:403] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:55.740144 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.740765 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.741122 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.759481 11398 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:55.743359 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.691715 13641 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.685634 13571 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.751127 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.698261 13833 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:55.764482 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.759769 11400 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:55.720631 13835 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:55.770730 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52032.2us
I20260709 13:15:55.764147 13641 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:15:55.776954 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.778067 11402 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:55.688359 13754 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.771373 13871 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.760780 13055 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.671356 13722 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.765858 13844 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.791997 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:55.770790 13763 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.794691 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.806066 11442 leader_election.cc:400] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:55.806541 11398 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:55.787417 13768 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.789405 13774 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.768543 13571 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.785385 13849 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.785984 13522 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.749493 13799 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:55.796603 13574 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.016s	user 0.006s	sys 0.005s
I20260709 13:15:55.807840 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:55.803570 13802 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:55.827968 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 64289.2us
I20260709 13:15:55.811198 13876 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:55.819169 13870 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:15:55.833802 13872 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:15:55.836153 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:55.837388 11473 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:55.835345 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.835680 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.835877 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.815486 13777 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.837697 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:55.842262 12122 raft_consensus.cc:2393] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:55.838831 11476 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:55.794617 13756 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.844277 13756 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:55.846577 13799 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.847254 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:15:55.847826 13868 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:15:55.781150 13856 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.848942 13571 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.849071 13879 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.849428 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:55.849922 13718 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.828509 13522 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:55.849700 13879 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.793296 13871 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.850397 13522 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.850463 13883 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.850896 13522 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.851003 13883 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.851475 13883 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.851804 13879 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:55.848982 13875 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:15:55.838846 11472 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:15:55.828195 13574 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.856752 11399 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:15:55.859133 13680 consensus_queue.cc:1175] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e9c2603c0bd4d25b713649b3890c46b" responder_term: 5 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:55.860641 12119 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:55.862020 13871 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.863276 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:55.863487 13755 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.864189 12113 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:55.762781 13869 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.876209 11472 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:55.876925 11476 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:55.878463 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.883836 13396 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.884112 13835 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.886709 13835 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.887180 11542 pending_rounds.cc:85] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 4
I20260709 13:15:55.887488 11542 pending_rounds.cc:107] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:15:55.887720 11542 raft_consensus.cc:2889] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:55.888886 13835 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:55.891050 13865 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:15:55.893443 13722 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.875738 13873 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.896273 13754 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.897452 13791 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:55.898586 13866 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 13:15:55.899158 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:55.867703 13763 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:55.900241 13763 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.902590 13763 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.867252 13876 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:55.903860 13876 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.918304 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.919354 11549 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:55.880043 13799 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.871912 13879 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.926692 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.934213 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.934814 11400 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.935182 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:55.936033 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:55.937462 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:55.938230 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:55.927353 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.928035 13884 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.928644 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.940618 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.940680 11545 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:55.928802 13775 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:55.929862 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:55.930212 13522 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.942557 11402 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:55.894541 13873 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.931164 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:55.943614 11474 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:15:55.935161 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:55.944873 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:55.945050 11400 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:55.946291 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:55.948351 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.945575 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:55.943852 13835 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:55.955211 13718 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:55.956213 13718 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.956635 13718 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.956710 13844 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.949692 13763 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:55.959597 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.956243 13835 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:55.959995 13763 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:55.960181 11402 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.960506 13763 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.960883 13835 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.942900 13606 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.963038 13606 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.963410 13606 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.926267 13799 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.965811 13522 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:55.955626 13886 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:55.942750 13845 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:55.966866 13886 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.967803 13845 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:55.968683 13845 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:55.969877 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.970045 13886 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.951444 13774 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.971338 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:55.971652 13774 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.972149 13774 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.972491 13889 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:55.975127 13718 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.977250 13718 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.978224 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:55.980186 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:55.981051 13839 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
I20260709 13:15:55.981509 13839 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.982580 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:55.982640 11549 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:55.982967 12085 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:55.985288 13722 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.985787 13722 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.955402 13869 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:55.988011 13722 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:55.988616 13722 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.989027 13722 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:55.990796 13861 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:55.991250 13861 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:55.991592 13861 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:55.992357 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1655) took 1390 ms. Trace:
I20260709 13:15:55.996093 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:55.997282 11393 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:55.998790 11398 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:15:56.000385 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:56.001731 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.002048 11398 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.992740 11468 rpcz_store.cc:276] 0709 13:15:54.601520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.603056 (+  1536us) service_pool.cc:224] Handling call
0709 13:15:54.621110 (+ 18054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.178724 (+557614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.992110 (+813386us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400213c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:55.992277 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1266666368}
I20260709 13:15:56.020323 11402 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:56.005596 11400 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:56.024394 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:56.024982 11551 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:56.028391 11398 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:56.025748 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.030648 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:56.020290 13589 consensus_meta.cc:287] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.006s
I20260709 13:15:56.033237 13882 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:15:56.036285 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.020133 13869 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:15:56.039060 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:55.988063 13644 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:56.036974 11474 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:56.041316 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:55.982689 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.040449 11402 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:56.053943 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.056821 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.057730 11475 raft_consensus.cc:2410] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:55.950425 13775 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.036389 13777 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:55.986333 13873 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.004114 13879 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.066392 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.066905 11475 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:56.067512 13756 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:56.042238 13644 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:56.039319 13718 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:55.950512 13884 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.029148 13889 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:15:56.062141 12116 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:56.074694 13722 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:56.075619 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.038825 13589 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.063876 13522 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:56.042055 13606 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.078157 13522 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:56.079762 13606 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.085971 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.068815 13777 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:56.089974 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.090381 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:56.091516 11469 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:56.099267 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.102265 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.103427 11474 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:56.075070 13644 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.117014 13644 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.081254 13861 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.124010 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.119539 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:56.124706 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.125084 11474 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:56.091460 13522 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.134407 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.134935 11472 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.135334 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.137120 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.137298 11474 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:56.139199 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:15:56.142508 13854 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.901s	user 0.000s	sys 0.006s
I20260709 13:15:56.148365 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.132299 13889 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:56.151947 13902 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.153759 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.154178 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.155731 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.157299 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.160791 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.155680 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.173664 12116 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:15:56.175597 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 136622us
I20260709 13:15:56.170512 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.145921 13722 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.176065 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.177590 11474 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:56.162205 13903 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.119102 13763 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.167593 13718 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.172120 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.180866 11475 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
W20260709 13:15:56.172610 13890 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:15:56.177716 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.164397 13889 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:56.183969 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.187083 11547 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:56.184077 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.186229 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.130718 13777 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.189150 11543 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 2.
I20260709 13:15:56.196619 11545 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:56.197438 13606 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.197626 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.198499 11545 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:56.199265 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.198076 13606 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.201411 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.180946 13839 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.220093 11545 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:56.184667 13879 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.191829 13889 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:56.244380 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:56.245440 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.252140 11547 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:15:56.253011 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 89570.4us
W20260709 13:15:56.254791 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 74701.6us
I20260709 13:15:56.200309 13886 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.200582 13778 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:56.256352 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.266201 11474 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:56.182976 13722 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.183704 13718 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.270502 11472 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:56.281168 13476 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.243172 13589 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.284821 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.285458 11395 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:56.286686 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:56.283524 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:56.291560 13775 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.293160 11547 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:56.329417 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75957.5us
I20260709 13:15:56.241833 13871 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:15:56.272671 13778 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:56.348568 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:56.351468 13891 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:15:56.355259 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70549.2us
I20260709 13:15:56.177002 13902 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.376034 13902 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.366569 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.348206 12115 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:56.363422 13855 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:56.354692 13778 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.186229 13903 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.273083 13396 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:15:56.371997 13589 raft_consensus.cc:993] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:15:56.411955 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.364244 13909 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:56.413467 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.413749 13909 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.414129 13909 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.200030 13883 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.332680 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.379480 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:56.353704 13861 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.345734 13884 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.421648 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:56.389065 13892 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:15:56.389173 13877 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.344887 13845 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.390625 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:56.391790 13606 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.392827 13825 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:56.392908 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1581) took 2126 ms. Trace:
I20260709 13:15:56.393594 13756 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:56.398463 13824 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.900s	user 0.011s	sys 0.000s
I20260709 13:15:56.399580 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:56.436573 11396 rpcz_store.cc:276] 0709 13:15:54.266464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.379492 (+113028us) service_pool.cc:224] Handling call
0709 13:15:55.685778 (+1306286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.689539 (+  3761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.392676 (+703137us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540035ded0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:56.392822 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3003438464}
W20260709 13:15:56.443259 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43781.6us
I20260709 13:15:56.385327 13571 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:56.407295 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51281.8us
I20260709 13:15:56.414098 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:56.284637 13886 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.445312 13571 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.429967 13899 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:56.440510 13896 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:15:56.434650 13825 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:56.496604 13718 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.441838 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.442507 12123 raft_consensus.cc:2368] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:15:56.417995 13396 raft_consensus.cc:1081] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:15:56.433565 12116 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:56.480806 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:56.444583 13335 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:56.444608 13908 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:56.430858 13590 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:56.419850 13589 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:56.469709 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 215226us
I20260709 13:15:56.503798 13722 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.503823 13589 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:56.506453 11396 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:56.482708 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 199218us
I20260709 13:15:56.430002 13846 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:56.467178 13883 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.507153 13846 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:56.507652 13846 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:56.480311 13774 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 13:15:56.469497 13763 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.470824 13904 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:56.441877 13845 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.506507 11399 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:56.469242 13873 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:15:56.513062 11469 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:56.492360 13571 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.468364 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:56.503573 13859 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.508692 11398 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:56.558238 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 242) took 2085 ms. Trace:
I20260709 13:15:56.484581 13756 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.511425 11441 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:56.551249 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.512986 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.551012 13335 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:56.555090 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:56.552258 11442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.3:46781 timed out after 2.158s (SENT) [suppressed 27 similar messages]
W20260709 13:15:56.557374 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 89305.6us
W20260709 13:15:56.567803 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 56474.3us
I20260709 13:15:56.499217 13899 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.497665 13750 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:56.442171 13877 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.569523 11441 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:56.571064 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 393) took 2421 ms. Trace:
I20260709 13:15:56.575263 13916 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:56.575865 13916 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:56.574795 12118 rpcz_store.cc:276] 0709 13:15:54.149333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.149531 (+   198us) service_pool.cc:224] Handling call
0709 13:15:54.149835 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:54.195570 (+ 45735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:54.195604 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:54.212519 (+ 16915us) log.cc:844] Serialized 52 byte log entry
0709 13:15:54.336633 (+124114us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:54.336675 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.336735 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:56.397908 (+2061173us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:56.561812 (+163904us) raft_consensus.cc:1685] finished
0709 13:15:56.561836 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:56.570958 (+  9122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17151,"raft.queue_time_us":16256,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":27860352,"thread_start_us":90577,"threads_started":1,"wal-append.queue_time_us":185043}
I20260709 13:15:56.576246 13916 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:56.558746 11544 rpcz_store.cc:276] 0709 13:15:54.467323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.479281 (+ 11958us) service_pool.cc:224] Handling call
0709 13:15:54.479665 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.532079 (+ 52414us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:54.532123 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.127743 (+1595620us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b2c000a0b68. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:15:56.128188 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.239849 (+111661us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:56.244803 (+  4954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.251595 (+  6792us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:56.514894 (+263299us) raft_consensus.cc:1685] finished
0709 13:15:56.514912 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:56.552112 (+ 37200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98352,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9081,"spinlock_wait_cycles":4220800,"thread_start_us":111102,"threads_started":1,"wal-append.queue_time_us":166262}
I20260709 13:15:56.550375 13722 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:56.579963 13722 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.567940 11474 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:56.587915 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.588182 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.588821 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:56.589279 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.589622 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.589812 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:56.590159 11400 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:15:56.590608 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.591359 11401 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:56.589062 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:56.592319 11395 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.594257 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.600976 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.567783 13522 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:56.608381 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:56.550235 13774 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.573146 13899 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.633270 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:56.571512 13859 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.636298 13722 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.617125 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.593194 13718 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:15:56.639816 13718 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:56.640179 13718 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:15:56.640705 13768 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.000s
I20260709 13:15:56.640805 13774 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.639422 13335 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.591974 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:56.633085 13775 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:56.507154 13589 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.607801 13777 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:56.633322 13522 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:56.641023 13718 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.496742 13825 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.643394 12112 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:56.657060 11398 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:56.583488 13750 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.616483 13883 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:56.616317 13861 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:15:56.641167 13768 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:56.670970 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.638255 13886 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.679934 13522 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.592034 13476 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:15:56.674577 13718 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:56.675132 13922 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:56.644713 13775 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.678678 12116 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:56.665232 13777 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.615574 13918 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:15:56.693082 13750 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.698431 11395 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:56.685271 13825 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:56.717868 13777 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:56.708180 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 155937us
I20260709 13:15:56.711925 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.718751 13877 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.673411 13883 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.717916 11477 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
W20260709 13:15:56.720036 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.712790 13845 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.723170 13877 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:56.724516 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.710314 13775 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:56.727386 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:56.732185 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:56.733939 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:56.734958 13914 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:56.726274 13883 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:56.743916 13901 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:15:56.744824 13680 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:56.747133 13917 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:56.747583 13917 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.715312 13889 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.733685 13768 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:56.804579 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64739.6us
I20260709 13:15:56.747942 13917 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.838699 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 476) took 1012 ms. Trace:
I20260709 13:15:56.839963 12119 rpcz_store.cc:276] 0709 13:15:55.826576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.827772 (+  1196us) service_pool.cc:224] Handling call
0709 13:15:55.828096 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.860957 (+ 32861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.838443 (+977486us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b54005e1ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:56.838634 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74113792}
I20260709 13:15:56.842077 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:56.845599 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:56.738466 13859 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.729851 13877 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.849140 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:56.852681 11471 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:56.854977 12119 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:56.858462 11394 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:15:56.858466 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.859408 11395 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:56.837690 13851 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.837497 13680 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.865154 13680 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.867060 13680 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.862211 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.863014 13843 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:56.836839 13914 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:56.870604 11471 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:56.723822 13916 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:56.885854 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:56.871397 13927 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:15:56.735026 13922 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:15:56.887590 13922 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:56.887936 13922 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:56.888712 13922 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.890201 13922 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:56.880167 13886 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:56.875548 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:56.882807 13680 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:56.867782 13899 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.885105 13889 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:56.896993 13791 consensus_queue.cc:1175] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "147201d22a694ca59f6e014f083a0892" responder_term: 5 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:56.905773 13763 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.907572 13884 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.853917 13768 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:56.909373 13843 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:56.920322 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:56.917881 13778 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.923094 13768 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:56.924021 13763 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.928671 13778 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.928946 13763 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:56.929490 13768 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:56.930583 13818 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:56.931882 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:56.933893 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:56.914657 13845 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:56.935534 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:56.935933 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:56.937855 11472 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:15:56.942456 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:56.944156 11549 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:15:56.919385 13680 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:56.913607 13571 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.294399 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 282) took 1368 ms. Trace:
I20260709 13:15:57.278476 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:57.303887 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.236274 11474 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:57.311563 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.312213 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:57.312314 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.310463 11393 rpcz_store.cc:276] 0709 13:15:55.926277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.927089 (+   812us) service_pool.cc:224] Handling call
0709 13:15:57.294213 (+1367124us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b5400349254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:57.294334 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131858688}
I20260709 13:15:57.314191 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:57.314471 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:15:57.023557 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.316545 11399 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:57.318120 13498 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.318608 13498 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.320701 13498 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.320932 13754 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.321470 13754 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.321938 13791 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:57.322511 13791 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.265313 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 397) took 2242 ms. Trace:
I20260709 13:15:57.324241 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1107) took 1488 ms. Trace:
W20260709 13:15:57.324855 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.324748 11477 rpcz_store.cc:276] 0709 13:15:55.836112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.836252 (+   140us) service_pool.cc:224] Handling call
0709 13:15:57.324062 (+1487810us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b540042a4d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:57.324164 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897746688}
I20260709 13:15:57.296154 13756 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.264063 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:56.946854 11472 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:57.312496 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:57.328622 11393 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:57.012728 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:57.330034 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.270958 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.331637 11550 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:56.989449 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:57.332360 11475 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:15:57.334414 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:57.071295 11548 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1651) took 2859 ms (client timeout 2158 ms). Trace:
I20260709 13:15:56.945853 11544 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:57.336364 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:56.970286 11476 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:15:57.338811 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:57.327576 13680 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.341956 13581 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.342675 13718 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.001s
I20260709 13:15:57.342862 13680 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.344707 13791 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.347615 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.348273 11547 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:15:57.298504 13846 consensus_meta.cc:287] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.790s	user 0.006s	sys 0.004s
I20260709 13:15:57.349080 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.352864 11547 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:57.349982 11551 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:15:57.350769 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 333168us
W20260709 13:15:57.335376 11548 rpcz_store.cc:269] 0709 13:15:54.212108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.212244 (+   136us) service_pool.cc:224] Handling call
0709 13:15:57.070917 (+2858673us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400206cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:57.071158 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2154300160}
I20260709 13:15:57.337378 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.323400 13589 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:57.161353 13887 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.098s	user 0.003s	sys 0.004s
I20260709 13:15:57.267686 13877 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:56.945542 13641 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.076290 13874 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:57.358371 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:56.936455 13843 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:57.323978 12117 rpcz_store.cc:276] 0709 13:15:55.023225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.025817 (+  2592us) service_pool.cc:224] Handling call
0709 13:15:55.103753 (+ 77936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.557811 (+454058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.264973 (+1707162us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400376ed4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:57.265230 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831207296}
I20260709 13:15:57.070535 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:57.362185 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 83779.6us
W20260709 13:15:56.971959 13909 consensus_meta.cc:287] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.004s
I20260709 13:15:57.377907 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:57.251183 13818 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:57.296432 13476 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:57.266971 13879 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:57.108464 13903 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.428036 11395 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:57.342703 13581 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.429121 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:57.532929 11402 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:15:57.384274 11401 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:57.218930 13899 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.534281 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.535884 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.536484 11544 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:57.437672 13874 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:15:56.987040 13644 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.870s	user 0.003s	sys 0.010s
I20260709 13:15:57.436354 11550 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:57.166296 13855 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.217s
I20260709 13:15:57.165333 13917 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:56.949951 13845 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.441892 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.474069 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.475684 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.024204 13777 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.382400 13791 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.382869 13922 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.936564 13884 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.383285 13867 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.383857 13718 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:57.529605 12114 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:57.537592 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:57.383733 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.584062 11546 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:57.554935 13778 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.378814 13590 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:15:57.531965 13476 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.478139 13763 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:57.533071 13775 consensus_meta.cc:287] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.795s	user 0.007s	sys 0.004s
I20260709 13:15:57.555969 11476 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:57.372464 13846 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.519923 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:57.432973 13641 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.560645 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1158) took 1821 ms. Trace:
I20260709 13:15:57.529840 13879 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:57.560925 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:57.564891 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.695514 11395 raft_consensus.cc:2368] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:15:57.521365 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:57.557713 13825 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.566357 13791 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.524893 13909 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.585139 11548 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:15:57.542330 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 184022us
I20260709 13:15:57.701417 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:57.557492 13833 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.590461 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:57.563119 13932 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.705703 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:57.581725 13851 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.711s	user 0.007s	sys 0.002s
I20260709 13:15:57.589001 13718 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:57.593461 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 484) took 1331 ms. Trace:
I20260709 13:15:57.589711 11551 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:57.530032 13818 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.595702 11515 leader_election.cc:400] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:15:57.730365 11515 leader_election.cc:403] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:57.527635 13916 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.572212 13755 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:57.678835 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.678959 13855 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.378602 13498 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:57.582705 13758 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:57.681253 11397 rpcz_store.cc:276] 0709 13:15:55.739458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.741649 (+  2191us) service_pool.cc:224] Handling call
0709 13:15:55.751251 (+  9602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:55.766299 (+ 15048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:55.766344 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:55.783937 (+ 17593us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.829292 (+ 45355us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:55.829331 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.829401 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.160746 (+1331345us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:57.554572 (+393826us) raft_consensus.cc:1685] finished
0709 13:15:57.554598 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.560573 (+  5975us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20666,"raft.queue_time_us":20498,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":5710,"spinlock_wait_cycles":4265216,"thread_start_us":20765,"threads_started":1,"wal-append.queue_time_us":155110}
I20260709 13:15:57.581933 13867 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.573426 13922 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.588066 13938 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:57.722901 11396 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:57.767269 11395 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:57.677115 13934 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.731065 13851 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.741011 13755 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:57.592864 13874 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.588776 13778 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.695389 13773 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.695092 13877 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:57.742257 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 146907us
I20260709 13:15:57.725430 13606 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:57.720256 13859 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.711445 12113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 481) took 1842 ms (client timeout 1940 ms). Trace:
I20260709 13:15:57.681599 13775 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:57.765028 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 200438us
W20260709 13:15:57.747290 13883 consensus_meta.cc:287] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.000s
I20260709 13:15:57.759326 13921 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:57.724399 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:57.588176 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 211337us
I20260709 13:15:57.836560 11548 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:57.695389 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.794169 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:57.737898 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.713166 13825 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.760258 13571 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:57.812124 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 274578us
I20260709 13:15:57.720438 12114 rpcz_store.cc:276] 0709 13:15:56.256174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.272200 (+ 16026us) service_pool.cc:224] Handling call
0709 13:15:57.070306 (+798106us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:57.587438 (+517132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4729216}
I20260709 13:15:57.823565 11395 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:57.845639 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.719017 13833 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.748481 13498 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:57.832485 11549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.035s	sys 0.047s
I20260709 13:15:57.860962 13498 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:57.862457 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.837766 11399 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:57.864102 13844 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:57.864117 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:57.865051 13844 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:15:57.865655 13844 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:57.866128 13844 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.867641 11544 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:57.868541 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.868731 13871 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:57.869103 13871 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:57.869609 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.870191 11544 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:57.869941 13871 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:57.837647 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.871178 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.816401 13606 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.718811 13932 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:57.845784 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51785.6us
I20260709 13:15:57.884328 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:57.809460 13856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.048s	sys 0.081s
I20260709 13:15:57.849080 13946 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:57.849707 13590 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.821612 12113 rpcz_store.cc:269] 0709 13:15:55.869373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.876405 (+  7032us) service_pool.cc:224] Handling call
0709 13:15:57.442201 (+1565796us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:15:57.711368 (+269167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24928000}
I20260709 13:15:57.853485 13909 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.812175 13396 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:15:57.855449 13922 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.856065 13641 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:57.856714 13846 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.762737 13758 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.796672 13934 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:57.859190 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:57.828373 13859 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.931607 11395 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:15:57.872169 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.753065 13855 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.839764 11546 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:57.846588 13916 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:15:57.891093 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:57.800812 13851 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.875186 13391 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.911346 13476 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.911458 13507 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:57.975690 11550 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:57.886821 13938 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:57.921342 13763 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:15:57.929831 13589 consensus_meta.cc:287] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:15:57.872596 13871 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.846061 13845 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.820902 13877 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:57.885268 13606 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:57.891839 13856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.049s	sys 0.081s
I20260709 13:15:57.828118 13773 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.004112 13938 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.005225 13574 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.006321 13773 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.006863 13773 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.007543 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:57.921685 13590 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.008217 13758 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.008747 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.009341 11472 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:58.010149 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.010321 13590 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.010649 11472 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:15:57.886533 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 180896us
W20260709 13:15:58.043061 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 152079us
I20260709 13:15:58.043303 13825 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.829581 13718 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.043428 13758 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.043793 11516 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:58.044996 11516 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.045295 13758 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:57.914077 13777 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.046092 13758 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.046837 13799 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:57.977437 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 162884us
I20260709 13:15:58.047771 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.048586 11395 raft_consensus.cc:2410] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:15:58.049429 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:57.920094 13917 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.050483 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.050758 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.051313 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.051213 13879 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.052526 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.052625 13845 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:58.052742 11367 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.053397 13778 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.053290 13845 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.053617 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.054003 13778 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.055387 13845 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.000025 11468 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:58.055859 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:57.871500 13921 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.056500 11551 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:58.057192 13754 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.058144 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.058625 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:15:58.067523 13680 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.003s
I20260709 13:15:58.083715 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.012477 13855 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.084307 13680 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.085028 13855 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.085497 13855 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.086428 13680 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.087041 13680 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:58.013906 13941 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:15:58.087394 13855 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.015918 13851 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.019070 13950 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.023837 13877 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.029528 13606 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.031196 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 245) took 2148 ms. Trace:
I20260709 13:15:58.088699 13950 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.089526 11542 rpcz_store.cc:276] 0709 13:15:55.882968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.883119 (+   151us) service_pool.cc:224] Handling call
0709 13:15:55.886950 (+  3831us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:55.888600 (+  1650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:55.888641 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:57.077409 (+1188768us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b2c000a0b68. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:15:57.078038 (+   629us) log.cc:844] Serialized 78 byte log entry
0709 13:15:57.389581 (+311543us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:57.465208 (+ 75627us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.471456 (+  6248us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.497560 (+ 26104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.497676 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:58.030826 (+533150us) raft_consensus.cc:1685] finished
0709 13:15:58.030857 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.031084 (+   227us) inbound_call.cc:177] Queueing success response
0709 13:15:58.044857 (+ 13773us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b18002fd3a0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
Metrics: {"mutex_wait_us":31469,"prepare.queue_time_us":77433,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":88,"raft.queue_time_us":58248,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":69814,"spinlock_wait_cycles":50014208,"thread_start_us":311077,"threads_started":1,"wal-append.queue_time_us":514866}
I20260709 13:15:58.032003 13396 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:58.100242 13950 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.103674 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1114) took 1170 ms. Trace:
I20260709 13:15:58.104055 11474 rpcz_store.cc:276] 0709 13:15:56.933267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.935485 (+  2218us) service_pool.cc:224] Handling call
0709 13:15:56.935758 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.311573 (+375815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.103417 (+791844us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54002a3854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:58.103610 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690066176}
I20260709 13:15:58.104907 13606 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.036280 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:57.967406 13948 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:15:58.106523 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 265409us
I20260709 13:15:58.041168 13791 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:57.841681 13883 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.046718 13758 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:58.055292 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.055938 13775 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:57.998075 13391 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.059399 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:58.059896 12122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 368) took 1519 ms (client timeout 1792 ms). Trace:
I20260709 13:15:58.122561 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.003938 13763 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.100245 13396 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:58.217680 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.060544 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:57.893215 13755 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:58.063277 13939 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.512s	user 0.003s	sys 0.005s
W20260709 13:15:58.063269 13953 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:15:58.238777 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 214806us
I20260709 13:15:58.043766 13574 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.063866 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.065367 13902 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:58.066116 11398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 290) took 1779 ms (client timeout 1541 ms). Trace:
I20260709 13:15:58.066888 11394 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:15:58.007383 13773 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.067917 11473 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:58.069164 13818 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.069722 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 293) took 1481 ms. Trace:
I20260709 13:15:57.981300 13867 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:57.995008 13916 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.260284 13916 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.260706 13391 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.261219 12114 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:58.262147 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.262327 13916 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.071472 13921 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.071256 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.072556 13581 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:58.073774 11471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 316) took 1935 ms (client timeout 2219 ms). Trace:
I20260709 13:15:58.263326 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:58.263599 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 192418us
W20260709 13:15:58.263542 11471 rpcz_store.cc:269] 0709 13:15:56.138828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.138960 (+   132us) service_pool.cc:224] Handling call
0709 13:15:58.073635 (+1934675us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.073724 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639524224}
I20260709 13:15:58.078226 13844 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.078881 12119 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:15:58.080006 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.265130 13554 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.265568 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:58.265731 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.265986 13883 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.266713 13883 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:58.266867 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.267170 13883 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.267678 12119 raft_consensus.cc:2368] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:58.081151 13917 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.268217 13917 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.268966 13958 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:58.081511 13876 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.082566 11442 leader_election.cc:400] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:15:58.271978 11442 leader_election.cc:403] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:15:58.272318 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:58.272697 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 190311us
I20260709 13:15:58.083222 13799 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.083784 13825 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.919397 13899 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.092260 11548 raft_consensus.cc:2368] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
I20260709 13:15:58.092971 11546 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:58.097702 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.098682 11472 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:58.276558 13899 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.277169 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.277580 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.278643 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.099618 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.279250 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.280175 11441 leader_election.cc:400] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:58.280330 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.280398 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.278707 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.281391 11551 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:58.052680 13884 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.105351 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.033042 13932 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.105808 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.106597 11477 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:58.280505 11441 leader_election.cc:403] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:15:58.040195 13922 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:58.106998 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.040567 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.103042 13859 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.114509 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.118500 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.120790 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1675) took 2089 ms. Trace:
I20260709 13:15:58.131656 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:58.130568 12122 rpcz_store.cc:269] 0709 13:15:56.540898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.652683 (+111785us) service_pool.cc:224] Handling call
0709 13:15:58.059732 (+1407049us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54006c0554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.059849 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2091209600}
I20260709 13:15:58.060073 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.288477 11401 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:58.229722 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.229722 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.107043 13680 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:58.222348 13843 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.002s
I20260709 13:15:58.250162 13949 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.253720 13936 raft_consensus.cc:993] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:58.122781 13606 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:15:58.254762 11396 rpcz_store.cc:276] 0709 13:15:56.588115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.588251 (+   136us) service_pool.cc:224] Handling call
0709 13:15:56.616467 (+ 28216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.767590 (+1151123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.069523 (+301933us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540035cfd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:58.069671 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544012160}
I20260709 13:15:58.257335 12113 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:15:58.258533 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.310348 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.311115 12117 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:58.313207 13680 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.314203 13606 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:58.314572 13773 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.259408 13874 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:58.315021 12112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 400) took 2162 ms (client timeout 1642 ms). Trace:
I20260709 13:15:58.264492 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.242080 13574 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.267608 13883 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.080323 13833 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.270921 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.317286 13574 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:58.320927 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50148.6us
I20260709 13:15:58.321034 13938 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.304337 12116 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:58.325711 13936 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:58.286159 13922 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:58.274152 13799 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.273638 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:15:58.329427 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 55900.1us
I20260709 13:15:58.329890 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 323) took 1009 ms. Trace:
I20260709 13:15:58.329473 13922 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.330153 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:58.330387 13799 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.330529 13922 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.300835 13884 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:58.306141 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.332114 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.332718 11475 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:58.333146 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1099) took 3760 ms. Trace:
I20260709 13:15:58.333725 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.334324 11475 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.334507 12118 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.334834 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.335530 13833 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.336220 13833 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:58.336567 13833 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.336905 13833 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.339135 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.339715 11395 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:58.340531 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.341010 11395 raft_consensus.cc:2376] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7. Re-sending same reply.
I20260709 13:15:58.341759 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.342674 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.272917 12123 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:58.248692 13902 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:58.288127 11468 rpcz_store.cc:276] 0709 13:15:56.031862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.031983 (+   121us) service_pool.cc:224] Handling call
0709 13:15:58.120558 (+2088575us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54002a4754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.120712 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4502786432}
W20260709 13:15:58.248847 11398 rpcz_store.cc:269] 0709 13:15:56.286303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.286437 (+   134us) service_pool.cc:224] Handling call
0709 13:15:58.065933 (+1779496us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.066046 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504458112}
I20260709 13:15:58.379427 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:58.291659 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 177217us
I20260709 13:15:58.293654 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.286240 13581 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.271914 13844 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:58.288753 13825 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.383742 13825 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.279382 13923 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:15:58.295915 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.384614 11472 raft_consensus.cc:2376] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8. Re-sending same reply.
I20260709 13:15:58.384790 12116 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:58.385531 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.386019 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.112032 13948 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.296756 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.387982 12122 raft_consensus.cc:2393] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:58.254798 13791 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.292204 13921 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.299113 13554 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.300486 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1121) took 1330 ms. Trace:
I20260709 13:15:58.303280 11477 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:15:58.299594 13949 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.393397 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 333) took 1040 ms. Trace:
W20260709 13:15:58.308406 12115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 362) took 2148 ms (client timeout 2074 ms). Trace:
W20260709 13:15:58.309091 13945 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.435s	user 0.006s	sys 0.000s
W20260709 13:15:58.317972 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53560.3us
W20260709 13:15:58.315478 12112 rpcz_store.cc:269] 0709 13:15:56.152183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.156946 (+  4763us) service_pool.cc:224] Handling call
0709 13:15:58.314803 (+2157857us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54006c0f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.314934 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077566208}
I20260709 13:15:58.319973 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.273990 13876 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.304744 13843 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.290386 13867 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:15:58.273860 13917 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.333515 11470 rpcz_store.cc:276] 0709 13:15:54.572267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.572380 (+   113us) service_pool.cc:224] Handling call
0709 13:15:54.575996 (+  3616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.576401 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:54.576428 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:54.576903 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.098773 (+1521870us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b18001c0060. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:15:56.132207 (+ 33434us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:56.273666 (+141459us) log.cc:844] Serialized 14 byte log entry
0709 13:15:56.343413 (+ 69747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.343574 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:58.330698 (+1987124us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b18001c0060. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:15:58.332937 (+  2239us) raft_consensus.cc:1685] finished
0709 13:15:58.332964 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.333075 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1680,"prepare.queue_time_us":48469,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":5016,"raft.queue_time_us":118693,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":299,"spinlock_wait_cycles":102244480,"thread_start_us":229893,"threads_started":3,"wal-append.queue_time_us":1620174}
I20260709 13:15:58.400700 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.401755 11470 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:58.351944 13498 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.314621 13606 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:58.399827 12117 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:58.403250 13606 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.404282 13876 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.404739 13606 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:58.394500 13791 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.407305 13791 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.358570 13883 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.330281 11476 rpcz_store.cc:276] 0709 13:15:57.320817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.338631 (+ 17814us) service_pool.cc:224] Handling call
0709 13:15:58.329728 (+991097us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540028bc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.329833 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497991936}
W20260709 13:15:58.373301 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.377744 11475 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:58.264457 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.380440 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.381002 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.386657 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.389621 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.389971 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.371827 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.387167 13935 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:58.417264 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45558.6us
I20260709 13:15:58.418205 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.392071 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.420037 13958 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:58.420113 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.420357 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.420743 13958 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:58.421129 13718 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:58.416245 13867 consensus_queue.cc:1243] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:58.421847 13884 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.307761 13818 raft_consensus.cc:1081] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:15:58.397333 13581 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:15:58.393132 11469 rpcz_store.cc:276] 0709 13:15:56.970259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.970485 (+   226us) service_pool.cc:224] Handling call
0709 13:15:56.970828 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:56.988228 (+ 17400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:56.988271 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.988865 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:15:57.356416 (+367551us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:57.356466 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.356546 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:58.062697 (+706151us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:58.287753 (+225056us) raft_consensus.cc:1685] finished
0709 13:15:58.287785 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.300427 (+ 12642us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2070,"raft.queue_time_us":26308,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":8984320,"thread_start_us":367189,"threads_started":1,"wal-append.queue_time_us":464873}
W20260709 13:15:58.398566 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 78700.2us
I20260709 13:15:58.425786 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.398593 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:58.392935 13844 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.412679 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.394634 13923 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:15:58.413343 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.429387 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.429623 11470 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.430011 11469 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:58.430210 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.415306 11400 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:58.430814 11476 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:58.419313 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.432086 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.420792 11397 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:58.421480 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.392130 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:58.423002 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.419986 12114 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:58.394862 11477 rpcz_store.cc:276] 0709 13:15:57.353351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.036009 (+682658us) service_pool.cc:224] Handling call
0709 13:15:58.393342 (+357333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.422313 13884 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.424475 11395 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.421559 13718 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:58.426615 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.435801 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.381053 13902 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:58.413031 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.426666 11399 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:58.440977 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.416721 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.396243 13921 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.442258 11475 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:58.437415 11397 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.439281 13758 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:58.451692 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 329) took 1129 ms. Trace:
I20260709 13:15:58.451779 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 330) took 1129 ms. Trace:
I20260709 13:15:58.453014 11471 rpcz_store.cc:276] 0709 13:15:57.322553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.451030 (+1128477us) service_pool.cc:224] Handling call
0709 13:15:58.451315 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.451471 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.451496 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.451537 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:58.451567 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.451624 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.451725 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.455271 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.395898 13554 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:15:58.436220 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44129us
I20260709 13:15:58.425040 13935 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:15:58.395946 12115 rpcz_store.cc:269] 0709 13:15:56.160272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.160461 (+   189us) service_pool.cc:224] Handling call
0709 13:15:58.308157 (+2147696us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54003778d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.308295 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293130624}
I20260709 13:15:58.409737 13825 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:58.436511 13775 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:58.436180 12083 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.441497 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.469471 13775 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:58.469791 12112 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:58.470429 13775 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.471153 13777 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:58.471701 13777 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:58.472545 13777 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.473397 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.448514 13948 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.473593 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.474848 13949 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:58.475144 13962 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:58.475266 12113 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:15:58.451614 13921 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.459913 13563 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:58.477767 13844 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.479514 11472 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.479837 13554 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.480075 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 332) took 1157 ms. Trace:
I20260709 13:15:58.480551 13843 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.480446 11472 rpcz_store.cc:276] 0709 13:15:57.328252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.479075 (+1150823us) service_pool.cc:224] Handling call
0709 13:15:58.479361 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.479872 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.480028 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.481488 11472 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:15:58.481858 11477 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:58.482012 11472 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:58.482527 11472 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.434737 13873 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:58.448450 13902 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:58.463330 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.464985 11469 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:15:58.465207 13935 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:15:58.466169 13933 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.627s	user 0.007s	sys 0.010s
I20260709 13:15:58.439589 13917 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.439980 13718 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.437134 13884 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.441679 13951 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:15:58.468068 13574 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.468672 11473 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:58.468724 13498 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.436403 13793 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:58.421631 13958 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.460144 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:58.483649 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46110.9us
I20260709 13:15:58.456095 11470 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:58.474928 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.485102 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:58.485226 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:58.485677 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49562.7us
I20260709 13:15:58.487829 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.452860 11475 rpcz_store.cc:276] 0709 13:15:57.322320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.450739 (+1128419us) service_pool.cc:224] Handling call
0709 13:15:58.451113 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.451331 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.451367 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.451420 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:58.451464 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.451531 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.451642 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.489900 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.468840 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.495959 11395 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:15:58.498078 11546 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:58.500072 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:58.571067 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110980us
I20260709 13:15:58.571918 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.546630 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:58.574529 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 86112.9us
I20260709 13:15:58.506541 11542 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:15:58.466775 13883 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.575229 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.582115 11475 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:15:58.582276 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.477795 13899 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.574668 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.542503 11397 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:15:58.576694 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.518733 11472 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.577317 11471 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.579057 11401 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:58.496639 13873 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.498963 13758 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:58.585457 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.545789 13917 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.521732 13563 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.586330 11396 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:15:58.586763 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.487941 13843 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:58.586599 13755 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.483237 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.595104 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:15:58.609261 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.607743 11472 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.515355 13955 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:15:58.587951 13923 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:58.612865 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.547703 13793 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.586654 13871 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:58.439651 13581 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.614187 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.635313 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43630 (request call id 1693) took 1290 ms. Trace:
I20260709 13:15:58.573415 12117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:15:58.570045 13948 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.569866 13938 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.624699 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.641420 11543 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:15:58.582798 13844 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.587860 13791 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:58.588414 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.618826 13843 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.606890 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.597059 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1695) took 1242 ms. Trace:
I20260709 13:15:58.628701 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.629789 11401 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:15:58.506753 13833 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.498456 13956 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.613710 13873 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.635783 11470 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.635807 11472 rpcz_store.cc:276] 0709 13:15:57.353633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.481365 (+1127732us) service_pool.cc:224] Handling call
0709 13:15:58.626239 (+144874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.636332 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.699581 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:58.613464 13949 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:58.637359 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 152309us
I20260709 13:15:58.732015 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:58.733381 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.613128 13877 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:58.630570 13874 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:58.626829 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.740285 11550 raft_consensus.cc:2376] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6. Re-sending same reply.
I20260709 13:15:58.741469 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.623744 13563 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.653836 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.743773 11546 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:58.695832 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67234.2us
I20260709 13:15:58.695876 11475 rpcz_store.cc:276] 0709 13:15:57.354158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.594579 (+1240421us) service_pool.cc:224] Handling call
0709 13:15:58.594932 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.596609 (+  1677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.596669 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.596734 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:58.596782 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.596868 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.596986 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.745558 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 327) took 1423 ms. Trace:
I20260709 13:15:58.745844 13884 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.746340 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1698) took 1391 ms. Trace:
I20260709 13:15:58.746814 11475 rpcz_store.cc:276] 0709 13:15:57.354604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.745487 (+1390883us) service_pool.cc:224] Handling call
0709 13:15:58.745823 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.746011 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.746037 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.746081 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:58.746112 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.746170 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.746280 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.748574 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1699) took 1393 ms. Trace:
I20260709 13:15:58.745960 11477 rpcz_store.cc:276] 0709 13:15:57.321857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.435008 (+1113151us) service_pool.cc:224] Handling call
0709 13:15:58.481535 (+ 46527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.482358 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.745341 (+262983us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b540039b2d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.745493 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98836352}
I20260709 13:15:58.749115 13884 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.748986 11475 rpcz_store.cc:276] 0709 13:15:57.354812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.747802 (+1392990us) service_pool.cc:224] Handling call
0709 13:15:58.748095 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.748265 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.748291 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.748334 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:58.748365 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.748417 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.748520 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.749871 11477 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.750633 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1700) took 1387 ms. Trace:
W20260709 13:15:58.697167 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.701193 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.635090 13793 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.651968 13843 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.699249 12115 raft_consensus.cc:2368] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8.
I20260709 13:15:58.743076 13554 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.648646 13938 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.751017 11477 rpcz_store.cc:276] 0709 13:15:57.362794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.749409 (+1386615us) service_pool.cc:224] Handling call
0709 13:15:58.749682 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.750363 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.750571 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.642899 13948 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.773840 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 334) took 1410 ms. Trace:
I20260709 13:15:58.649561 13844 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.775154 11550 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:15:58.647370 12117 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:58.750941 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1701) took 1387 ms. Trace:
I20260709 13:15:58.707304 13956 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.710150 13917 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.715682 11468 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:58.716933 13833 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.715708 13574 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.728224 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.734915 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.774241 11477 rpcz_store.cc:276] 0709 13:15:57.363304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.772997 (+1409693us) service_pool.cc:224] Handling call
0709 13:15:58.773300 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.773488 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.773513 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.773552 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:58.773580 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.773638 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.773762 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.784721 11548 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
W20260709 13:15:58.785420 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50623.6us
I20260709 13:15:58.785735 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1702) took 1422 ms. Trace:
I20260709 13:15:58.755131 13563 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:58.786110 11477 rpcz_store.cc:276] 0709 13:15:57.363367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.784925 (+1421558us) service_pool.cc:224] Handling call
0709 13:15:58.785241 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.785415 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.785439 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.785487 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:58.785523 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.785577 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.785681 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.739776 13877 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.741947 13874 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.791698 11470 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:58.776269 11475 rpcz_store.cc:276] 0709 13:15:57.363092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.750147 (+1387055us) service_pool.cc:224] Handling call
0709 13:15:58.750437 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.750598 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.750622 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.750675 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:58.750706 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.750768 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.750883 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.755246 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.756212 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.756973 11545 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.780450 12117 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.755579 13879 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.752789 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:58.799614 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 225113us
I20260709 13:15:58.695813 13606 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.729283 13923 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.757433 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.801203 11551 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.794366 11549 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.794814 11547 raft_consensus.cc:2393] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:15:58.793911 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1703) took 1430 ms. Trace:
I20260709 13:15:58.796402 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.803745 11475 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:58.803503 11477 rpcz_store.cc:276] 0709 13:15:57.372925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.788486 (+1415561us) service_pool.cc:224] Handling call
0709 13:15:58.788786 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.793546 (+  4760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.793573 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.793638 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:58.793690 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.793746 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.793860 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.796970 11545 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.806239 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.769290 13938 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:58.800097 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47646.4us
I20260709 13:15:58.812340 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.792059 13554 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:58.807719 11477 raft_consensus.cc:2368] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5.
W20260709 13:15:58.796104 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 214169us
I20260709 13:15:58.820922 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:15:58.811496 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.793876 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.813733 11547 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:15:58.813675 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:58.828382 11470 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:15:58.827054 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:58.808095 13949 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:58.793694 13877 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.824883 11366 leader_election.cc:400] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8.
I20260709 13:15:58.833258 11366 leader_election.cc:403] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:15:58.794044 13874 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.826026 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1704) took 1452 ms. Trace:
I20260709 13:15:58.834182 11477 rpcz_store.cc:276] 0709 13:15:57.373389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.825200 (+1451811us) service_pool.cc:224] Handling call
0709 13:15:58.825516 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.825694 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.825720 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.825765 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:58.825818 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.825869 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.825972 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.834839 11365 leader_election.cc:400] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5.
I20260709 13:15:58.832495 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.766103 13758 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.838321 11365 leader_election.cc:403] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:15:58.812791 13606 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.835358 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:58.836208 11546 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:15:58.752912 13396 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.820470 13833 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:58.878453 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.833675 13849 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.828904 13554 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:58.846505 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:58.839551 13926 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.842739 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:58.835990 13923 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.892123 13923 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:58.892582 13923 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.892284 13554 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.842831 13917 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:58.859227 13752 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5.
I20260709 13:15:58.837853 13862 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.860407 11470 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:58.895942 13862 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.896230 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1706) took 1522 ms. Trace:
I20260709 13:15:58.896740 11470 rpcz_store.cc:276] 0709 13:15:57.374006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.840586 (+1466580us) service_pool.cc:224] Handling call
0709 13:15:58.860234 (+ 19648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.895965 (+ 35731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.896164 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.897904 13862 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.860538 13844 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.861756 13956 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.826925 13867 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:15:58.869316 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:58.871454 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 377151us
I20260709 13:15:58.872862 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.903990 11548 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.904068 12085 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:58.873557 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.877148 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:58.875861 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.904902 11547 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:58.807663 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:58.905699 11475 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.883533 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.906937 11543 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:58.887344 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.908727 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.908567 13843 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.880527 13972 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:15:58.898663 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1707) took 1524 ms. Trace:
I20260709 13:15:58.910334 11470 rpcz_store.cc:276] 0709 13:15:57.374299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.897654 (+1523355us) service_pool.cc:224] Handling call
0709 13:15:58.898037 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.898242 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.898282 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.898356 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:58.898399 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.898464 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.898600 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.911780 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.912253 13934 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:58.912902 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.913178 13867 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:58.913565 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:58.909890 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.914198 13971 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:58.914351 11550 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.914744 11402 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:58.914852 13971 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:58.915613 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.915853 13971 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.918119 11367 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.918951 13574 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:58.919113 13827 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:58.920042 13827 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:58.916750 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.921432 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.922111 11473 raft_consensus.cc:2368] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:58.922212 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.909744 11401 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:58.924172 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47110.3us
I20260709 13:15:58.913658 13867 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:58.924551 12083 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:58.932458 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 326) took 1610 ms. Trace:
I20260709 13:15:58.932950 11469 rpcz_store.cc:276] 0709 13:15:57.321599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.430629 (+1109030us) service_pool.cc:224] Handling call
0709 13:15:58.430886 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.538499 (+107613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.932116 (+393617us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b540044b1d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:58.932383 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77683072}
W20260709 13:15:58.935413 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1173) took 1621 ms (client timeout 1792 ms). Trace:
I20260709 13:15:58.939168 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 331) took 1616 ms. Trace:
I20260709 13:15:58.939687 11471 rpcz_store.cc:276] 0709 13:15:57.322779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.454180 (+1131401us) service_pool.cc:224] Handling call
0709 13:15:58.577134 (+122954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.590166 (+ 13032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.938902 (+348736us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540028a350. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.939095 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281984768}
I20260709 13:15:58.941694 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 335) took 1562 ms. Trace:
I20260709 13:15:58.942116 11471 rpcz_store.cc:276] 0709 13:15:57.379429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.940738 (+1561309us) service_pool.cc:224] Handling call
0709 13:15:58.941092 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.941308 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.941347 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.941403 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:58.941446 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.941514 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.941632 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.925005 11473 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.950199 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1708) took 1575 ms. Trace:
I20260709 13:15:58.890712 13818 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.881175 13849 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.926026 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:58.913605 11470 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:58.913443 13777 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:15:58.943982 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 107650us
I20260709 13:15:58.948750 11548 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:58.923460 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:58.886909 13948 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:58.924932 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.899092 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.882584 13871 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:15:58.955639 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:58.950675 11473 rpcz_store.cc:276] 0709 13:15:57.374582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.922617 (+1548035us) service_pool.cc:224] Handling call
0709 13:15:58.922894 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.949956 (+ 27062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.950147 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.957485 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.957831 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.958603 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.949481 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:15:58.935913 11400 rpcz_store.cc:269] 0709 13:15:57.314440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.329725 (+ 15285us) service_pool.cc:224] Handling call
0709 13:15:58.935218 (+1605493us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003d4ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:58.935348 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493949824}
I20260709 13:15:58.986222 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:58.988041 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:58.978542 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52599.3us
I20260709 13:15:58.962735 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.980495 13827 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:58.981069 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56215.4us
I20260709 13:15:58.982555 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:58.991673 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:58.996404 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.996989 11471 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:58.997997 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:58.968581 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.000447 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.001650 11550 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:59.007444 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.010946 11471 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:15:58.950198 13844 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.012260 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.013293 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:58.977389 13849 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.913228 13718 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:58.953652 13956 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:58.961289 13883 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:58.953608 13902 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:15:59.016683 11367 leader_election.cc:400] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
I20260709 13:15:59.038349 11367 leader_election.cc:403] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:15:59.039316 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48872us
I20260709 13:15:59.006469 11475 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:59.018457 11400 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:59.015165 11470 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:58.954471 13818 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:58.985814 13777 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:59.010324 13968 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:15:59.017786 13938 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:59.027663 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.006857 13948 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.037544 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.016261 13845 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:58.964839 13874 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:58.996053 13498 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:59.055193 13883 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:59.020622 13767 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:59.065907 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:59.076742 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:59.057219 13936 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:59.067891 11470 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:59.053130 13818 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:59.071669 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.066695 11475 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:59.083992 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:59.053305 13777 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.079159 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.087167 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.088032 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 328) took 1766 ms. Trace:
W20260709 13:15:59.088599 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 136807us
I20260709 13:15:59.088531 11476 rpcz_store.cc:276] 0709 13:15:57.322084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.443074 (+1120990us) service_pool.cc:224] Handling call
0709 13:15:58.465793 (+ 22719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.486128 (+ 20335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.486166 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:58.486596 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:15:58.624234 (+137638us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:58.624539 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.624781 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.624854 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:59.087836 (+462982us) raft_consensus.cc:1685] finished
0709 13:15:59.087855 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.087973 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":3275,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7730,"thread_start_us":137311,"threads_started":1,"wal-append.queue_time_us":338293}
I20260709 13:15:59.028713 13793 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.093976 11468 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:59.038578 13956 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.065301 13845 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:59.060858 12113 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:59.094244 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.075773 13862 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:15:59.071998 13913 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
I20260709 13:15:59.117115 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.094686 13949 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.122879 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:59.097019 13791 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.124004 13791 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.092730 13833 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:59.127115 13883 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.128386 13833 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:59.129552 13833 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.059531 13867 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.106269 13606 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.115489 13845 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.117086 13799 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.140296 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:59.098282 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1709) took 1723 ms. Trace:
I20260709 13:15:59.065157 13938 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:59.144889 13938 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.145897 13874 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.068507 13859 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.146761 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1185) took 1072 ms. Trace:
W20260709 13:15:59.099655 13975 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:59.085774 13934 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:59.149752 13934 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:59.150197 13934 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:59.079797 13775 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:15:59.100710 13843 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.101722 13923 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.151859 13936 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:59.152432 13936 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.102276 13956 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.085985 13917 raft_consensus.cc:2764] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:59.105543 13793 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.154939 11471 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:59.113468 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.156188 13917 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.126698 13791 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.159902 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:15:59.161659 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.100386 13958 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:59.116007 11549 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:15:59.145275 13498 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:59.153645 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:15:59.162701 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 49337.9us
I20260709 13:15:59.102571 13873 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.166004 11469 rpcz_store.cc:276] 0709 13:15:57.374876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.934059 (+1559183us) service_pool.cc:224] Handling call
0709 13:15:58.934446 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.934688 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:58.979157 (+ 44469us) log.cc:844] Serialized 14 byte log entry
0709 13:15:59.097860 (+118703us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:59.097909 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:59.097952 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.098011 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.098116 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45258,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":76,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6044,"thread_start_us":118357,"threads_started":1}
I20260709 13:15:59.163982 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.165887 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.147203 11396 rpcz_store.cc:276] 0709 13:15:58.074751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.618057 (+543306us) service_pool.cc:224] Handling call
0709 13:15:58.639623 (+ 21566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.649672 (+ 10049us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:58.649710 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:58.695287 (+ 45577us) log.cc:844] Serialized 26 byte log entry
0709 13:15:59.053533 (+358246us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:59.053598 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.053694 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:59.146531 (+ 92837us) raft_consensus.cc:1685] finished
0709 13:15:59.146553 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.146701 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3166,"raft.queue_time_us":3348,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":272795,"threads_started":1,"wal-append.queue_time_us":355781}
I20260709 13:15:59.170584 11549 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:15:59.169844 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:59.171258 11550 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.171746 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.172885 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.174015 13680 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.172612 11396 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:59.176753 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.178403 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.179853 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.180557 11396 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:59.174808 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.181735 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.188484 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.188923 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.181807 13455 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:59.182770 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 306) took 1105 ms. Trace:
I20260709 13:15:59.185223 11550 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:15:59.204656 11401 rpcz_store.cc:276] 0709 13:15:58.077192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.695518 (+618326us) service_pool.cc:224] Handling call
0709 13:15:58.704751 (+  9233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.924432 (+219681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.182542 (+258110us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540030aa54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:59.182715 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471010688}
I20260709 13:15:59.185477 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.205950 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.206521 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.186453 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:15:59.114171 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 123304us
I20260709 13:15:59.203243 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:15:59.208505 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.209724 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1710) took 1668 ms. Trace:
I20260709 13:15:59.210377 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.210098 11468 rpcz_store.cc:276] 0709 13:15:57.541250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.946123 (+1404873us) service_pool.cc:224] Handling call
0709 13:15:58.978363 (+ 32240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.095520 (+117157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.209538 (+114018us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002a4c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:59.209676 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265345408}
I20260709 13:15:59.212421 13917 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.212914 13917 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.214936 13917 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.218729 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:15:59.219354 11477 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:59.222450 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.223510 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1608) took 2340 ms. Trace:
I20260709 13:15:59.223974 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.224361 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.224700 11543 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:15:59.224876 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.225528 11475 raft_consensus.cc:2393] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:59.225476 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.168627 13498 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:59.228456 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.223934 11394 rpcz_store.cc:276] 0709 13:15:56.862270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.862443 (+   173us) service_pool.cc:224] Handling call
0709 13:15:58.066667 (+1204224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.249826 (+183159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.202128 (+952302us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540035ded0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:59.202259 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2768591104}
I20260709 13:15:59.228886 11399 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:59.227758 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.168799 13718 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:15:59.239768 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50918.2us
I20260709 13:15:59.201499 13680 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.231909 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.234526 11514 leader_election.cc:400] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:59.235446 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.245499 11514 leader_election.cc:403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:15:59.226209 13755 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 13:15:59.229496 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.237448 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.185026 13843 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:59.207841 13791 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.249634 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.204550 13455 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:59.231171 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.253489 11474 raft_consensus.cc:2410] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:15:59.250623 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:15:59.250912 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 160651us
I20260709 13:15:59.247170 11396 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.228042 13407 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.230234 13923 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.266011 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.224614 13948 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:15:59.236709 13763 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.241703 13159 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:59.257336 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.253343 13775 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:59.292130 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.253253 13791 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:15:59.295672 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62111.8us
I20260709 13:15:59.302611 11470 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:59.255599 13949 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.262688 13843 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.266314 13755 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.294293 13756 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.300741 13923 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:59.296020 13859 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:59.268636 13407 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.302008 13791 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.310422 13926 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8.
I20260709 13:15:59.267781 13984 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:15:59.336184 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1697) took 1981 ms. Trace:
W20260709 13:15:59.339445 13879 consensus_meta.cc:287] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
W20260709 13:15:59.340128 13877 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.001s
I20260709 13:15:59.312942 13455 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.330440 13923 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:15:59.344233 13407 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.332844 13859 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.344888 11401 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.344911 11473 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:59.338512 13849 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.327440 13755 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.310019 13159 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:59.351482 11472 rpcz_store.cc:276] 0709 13:15:57.354466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.737974 (+1383508us) service_pool.cc:224] Handling call
0709 13:15:58.738472 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.868096 (+129624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.868128 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.868171 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:58.868201 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.335887 (+467686us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400213754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:59.335942 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.336122 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297635072}
I20260709 13:15:59.363163 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.388067 11393 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:59.368103 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.326879 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:59.370222 13828 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:15:59.383391 12117 consensus_meta.cc:287] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:15:59.397563 11401 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:15:59.393194 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:15:59.382567 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 78428us
I20260709 13:15:59.328249 13843 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.380481 13923 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.378250 13877 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.403268 13877 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.404296 11475 raft_consensus.cc:2368] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:15:59.385306 13859 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.400107 11395 raft_consensus.cc:2368] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
W20260709 13:15:59.399184 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99086.9us
I20260709 13:15:59.385020 13755 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.406415 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:59.391389 13756 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:59.416219 12116 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.330017 13949 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.416425 13159 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.399459 13455 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.407068 12117 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.436894 13950 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:59.437860 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 207492us
I20260709 13:15:59.438495 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1175) took 2091 ms. Trace:
I20260709 13:15:59.439057 13936 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.439707 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.440753 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.441401 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.441794 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:59.442416 13921 raft_consensus.cc:2764] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:59.443372 12112 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:59.438882 11393 rpcz_store.cc:276] 0709 13:15:57.347468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.348986 (+  1518us) service_pool.cc:224] Handling call
0709 13:15:57.349274 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.388503 (+2039229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.438239 (+ 49736us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54001e00d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:15:59.438434 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4667609856}
W20260709 13:15:59.444077 13991 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:15:59.444399 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.441797 13391 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:59.445089 13758 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:59.445178 13606 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.442662 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.446010 13954 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:59.445250 13763 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:59.406884 13982 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:15:59.446885 11393 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:59.447079 13763 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:15:59.447570 13763 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.447853 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.448554 11393 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:59.449357 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.458452 12116 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:15:59.460486 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 212669us
I20260709 13:15:59.450173 11393 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
W20260709 13:15:59.465952 11474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1694) took 2112 ms (client timeout 2042 ms). Trace:
W20260709 13:15:59.466435 11474 rpcz_store.cc:269] 0709 13:15:57.353937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.105078 (+751141us) service_pool.cc:224] Handling call
0709 13:15:59.465730 (+1360652us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b5400214154. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:59.465887 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2231191936}
I20260709 13:15:59.479048 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:15:59.479717 11401 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:15:59.484253 13992 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:15:59.412883 13989 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:15:59.443214 13777 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.486917 13777 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:59.487447 13973 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.391s	user 0.004s	sys 0.000s
I20260709 13:15:59.488667 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1180) took 1415 ms. Trace:
I20260709 13:15:59.489030 11397 rpcz_store.cc:276] 0709 13:15:58.073507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.617332 (+543825us) service_pool.cc:224] Handling call
0709 13:15:58.621889 (+  4557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.622181 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.622224 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:58.622846 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:15:58.984819 (+361973us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:58.984858 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.984929 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:59.430979 (+446050us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:15:59.488320 (+ 57341us) raft_consensus.cc:1685] finished
0709 13:15:59.488342 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.488594 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6102784,"thread_start_us":315764,"threads_started":1,"wal-append.queue_time_us":418146}
I20260709 13:15:59.462664 13758 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:15:59.491288 13758 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.492079 11396 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:15:59.492965 13996 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:15:59.493613 13758 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:59.461340 13843 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.492902 13998 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.477496 13934 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.494884 13849 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.495949 13859 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.496472 13934 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.497859 13859 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.500548 13818 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.502024 11400 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:15:59.503558 13923 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.505385 11400 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:59.505661 13923 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.506669 11476 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:59.507275 13995 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:15:59.399597 13828 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.510169 13828 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:15:59.511902 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50243.7us
I20260709 13:15:59.503775 13997 consensus_queue.cc:1059] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:59.513127 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.513273 13997 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:59.513674 13997 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:15:59.514611 13997 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.516144 13997 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:59.445130 13391 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:59.523851 13391 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.525439 13391 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.530279 13982 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.530735 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:59.531579 13775 raft_consensus.cc:2764] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:15:59.493639 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 315) took 1140 ms. Trace:
I20260709 13:15:59.532001 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:15:59.531651 13873 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.532837 13775 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:59.532325 11396 rpcz_store.cc:276] 0709 13:15:58.353025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.492824 (+1139799us) service_pool.cc:224] Handling call
0709 13:15:59.493114 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.493337 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.493361 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:59.493403 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:59.493431 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.493480 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.493573 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:59.535713 13845 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.536434 11396 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:59.537966 13845 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:15:59.540906 13763 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.441040 13936 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.544246 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:15:59.544327 11395 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:59.547981 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41722
I20260709 13:15:59.575726 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:59.581410 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:15:59.579880 13777 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:59.584476 13777 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:59.585527 13777 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.596908 11516 leader_election.cc:400] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:15:59.597357 11516 leader_election.cc:403] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:59.592375 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.600996 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.602212 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.606418 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1189) took 1378 ms. Trace:
I20260709 13:15:59.608474 11468 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:59.609064 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:15:59.606863 11395 rpcz_store.cc:276] 0709 13:15:58.227924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.464594 (+1236670us) service_pool.cc:224] Handling call
0709 13:15:59.544079 (+ 79485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.548178 (+  4099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.606219 (+ 58041us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b54003d3bd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:15:59.606359 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181995776}
W20260709 13:15:59.618158 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 207992us
I20260709 13:15:59.569485 13921 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 13:15:59.659543 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48245us
W20260709 13:15:59.659466 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43955.2us
I20260709 13:15:59.665333 11395 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:59.629279 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.671011 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.662416 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.612957 13758 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.673481 11476 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:59.593415 13899 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:59.620258 13756 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:59.619844 13917 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:59.675906 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.658133 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:15:59.756917 13982 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:59.676671 13999 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:15:59.676422 14006 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.805110 14006 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:15:59.807547 14006 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:15:59.677157 14002 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:59.677204 13879 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:15:59.736943 13954 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:15:59.599260 13845 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.154s
W20260709 13:15:59.815234 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.671582 13987 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.186s
I20260709 13:15:59.764118 11395 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.614042 13859 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.619287 13844 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.142s
W20260709 13:15:59.841023 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 165575us
I20260709 13:15:59.672739 13763 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:15:59.853060 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 442505us
I20260709 13:15:59.863896 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.868909 11544 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:15:59.837263 13899 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:15:59.873478 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.874293 11545 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:15:59.737237 13989 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.177s
I20260709 13:15:59.835376 13756 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:59.834653 13859 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:59.876302 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 434555us
I20260709 13:15:59.889631 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:15:59.835844 13917 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:15:59.895282 11545 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:59.896859 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:59.899639 11395 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:15:59.910756 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:59.870528 13845 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:59.917672 11542 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:59.912009 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.914495 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:15:59.914721 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:15:59.915017 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:15:59.872665 13879 consensus_queue.cc:1243] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:15:59.872958 13923 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:59.920871 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57338.5us
I20260709 13:15:59.951443 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.951838 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.952586 11475 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.953212 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.954231 11473 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:15:59.951351 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.894937 13763 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.922865 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.924567 13641 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:59.925314 13563 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:59.935238 13934 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:59.940656 14010 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.959347 11468 raft_consensus.cc:2410] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:15:59.961158 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.905233 13859 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:59.968896 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 294431us
I20260709 13:15:59.974673 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:15:59.983522 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.917254 13845 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:15:59.986044 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:15:59.987371 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:15:59.903324 13755 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:15:59.991911 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.958693 13877 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:15:59.994607 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:15:59.988809 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:00.084229 11473 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:00.084362 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 95635.2us
I20260709 13:15:59.915892 13956 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:15:59.980799 13680 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:00.085440 13680 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.086000 13883 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.984313 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:00.086753 11395 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.973742 13818 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.056326 11475 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:15:59.984282 13498 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.739s	user 0.008s	sys 0.009s
I20260709 13:15:59.916832 13954 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:16:00.104154 11473 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:00.099946 13498 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.104802 13934 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:00.106245 11470 raft_consensus.cc:2368] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7.
I20260709 13:16:00.168980 11472 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:16:00.115336 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 128072us
I20260709 13:16:00.121295 13972 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:00.132690 13755 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:15:59.973912 13763 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.134668 14008 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:00.138676 13396 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:00.118672 13791 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.000s
W20260709 13:16:00.096323 13997 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.006s
I20260709 13:16:00.161808 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:00.164775 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 236749us
I20260709 13:16:00.165062 11395 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:15:59.972146 13899 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.186511 13989 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:16:00.193879 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.194674 13680 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.195636 13498 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.196336 13934 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.170096 13954 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:00.179613 14008 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:00.181973 13641 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.109753 13756 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.204439 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:00.170089 13984 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:00.206961 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 223527us
I20260709 13:16:00.175563 13972 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:00.160801 13917 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.187891 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:00.208442 13921 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:00.191995 13791 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.187345 13877 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.242508 11473 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.230530 12112 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:00.247654 13791 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:00.221362 13641 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:00.252596 13758 consensus_meta.cc:287] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:16:00.253069 13758 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:00.218566 13954 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.260773 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1188) took 2130 ms. Trace:
I20260709 13:16:00.261693 11550 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:00.261262 11398 rpcz_store.cc:276] 0709 13:15:58.130287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.244663 (+1114376us) service_pool.cc:224] Handling call
0709 13:15:59.244955 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.257003 (+ 12048us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:59.407297 (+150294us) log.cc:844] Serialized 14 byte log entry
0709 13:15:59.662709 (+255412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:59.662781 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:59.662843 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.260527 (+597684us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b540019a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:00.260580 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.260704 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153536,"prepare.queue_time_us":170107,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":35883,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":60350,"spinlock_wait_cycles":27219840,"thread_start_us":339688,"threads_started":2,"wal-append.queue_time_us":496904}
I20260709 13:16:00.265558 11398 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.268316 11398 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:00.273635 11398 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:00.276327 11398 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:00.252944 13641 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:00.260144 14007 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:16:00.226946 13984 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.280295 13954 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:00.285956 13877 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:00.287122 11394 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:16:00.293692 11394 pending_rounds.cc:85] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 6
I20260709 13:16:00.293941 11394 pending_rounds.cc:107] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:16:00.294196 11394 raft_consensus.cc:2889] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:00.260301 13763 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:00.294615 13984 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.236290 13921 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.296589 13921 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:00.290776 14025 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:00.279263 13972 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.292042 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 343) took 1031 ms. Trace:
I20260709 13:16:00.307667 11469 rpcz_store.cc:276] 0709 13:15:59.261017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.261203 (+   186us) service_pool.cc:224] Handling call
0709 13:15:59.261612 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.262046 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.262075 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:59.295157 (+ 33082us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.118255 (+823098us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b180028aaa0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:00.182127 (+ 63872us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.182168 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.182242 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:00.291708 (+109466us) raft_consensus.cc:1685] finished
0709 13:16:00.291730 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.291965 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":23822,"spinlock_wait_cycles":5534464,"thread_start_us":63596,"threads_started":1,"wal-append.queue_time_us":972038}
I20260709 13:16:00.305825 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.311108 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.313480 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.315860 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.317322 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.324924 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:00.301756 13999 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.332504 12112 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
W20260709 13:16:00.315601 13554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.273s	user 0.007s	sys 0.033s
I20260709 13:16:00.314774 13877 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:00.319847 13680 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:00.350541 13680 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.351161 13680 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.316463 13791 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:00.335381 13554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.007s	sys 0.033s
I20260709 13:16:00.353789 13791 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:00.354148 11395 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:00.354308 13791 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.358523 13791 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.335665 13999 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:00.326411 13972 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:00.337179 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 112222us
I20260709 13:16:00.340622 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:00.342132 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.369000 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.328222 13756 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:00.328200 14030 raft_consensus.cc:2764] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:00.344606 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.370126 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:00.345332 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.345767 11545 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.345955 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.371032 11543 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:16:00.244642 14008 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.376618 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.352741 13859 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.319960 13755 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:00.355265 13680 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:00.355829 13949 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:00.356221 11395 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:00.377593 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:00.386914 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:00.378969 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:00.379352 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 192887us
I20260709 13:16:00.379791 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 312) took 2075 ms. Trace:
I20260709 13:16:00.380162 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:00.380472 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:00.381101 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:00.381717 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:00.382371 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:00.382916 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:00.378537 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 463863us
I20260709 13:16:00.385502 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:00.411934 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:00.412788 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:00.414188 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:00.432138 14008 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:00.401961 13818 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:00.416023 14010 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:16:00.441743 11397 rpcz_store.cc:276] 0709 13:15:58.304006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.490465 (+1186459us) service_pool.cc:224] Handling call
0709 13:15:59.490834 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:59.549053 (+ 58219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.549084 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:59.573487 (+ 24403us) log.cc:844] Serialized 52 byte log entry
0709 13:15:59.984587 (+411100us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.191160 (+206573us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.379489 (+188329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.379559 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:00.379599 (+    40us) raft_consensus.cc:1685] finished
0709 13:16:00.379612 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.379737 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266767,"prepare.queue_time_us":135678,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":312359,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":2818,"thread_start_us":572294,"threads_started":2,"wal-append.queue_time_us":625174}
I20260709 13:16:00.440227 11543 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:00.442209 12112 raft_consensus.cc:2368] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
W20260709 13:16:00.442608 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 62207.1us
I20260709 13:16:00.385562 13791 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:00.385311 13563 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:00.443138 12116 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:00.445842 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:00.444186 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:00.446559 11544 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:00.455255 11473 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:16:00.385547 14012 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:00.473089 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:00.441599 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:00.474385 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:00.474953 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:00.443499 12122 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:16:00.471002 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61961.7us
I20260709 13:16:00.434669 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:00.434170 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:00.491984 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 115446us
I20260709 13:16:00.440966 14022 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:00.434356 13680 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:00.494755 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.470079 14030 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:00.485474 13606 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:00.413347 13956 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:16:00.459892 13877 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:00.502328 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 123416us
W20260709 13:16:00.484232 14029 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:16:00.460264 13818 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:00.470813 13833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.009s	sys 0.027s
I20260709 13:16:00.411284 13758 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:16:00.512529 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:00.513744 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:00.488364 13935 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:00.520110 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1192) took 2246 ms. Trace:
I20260709 13:16:00.489408 14010 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:00.491114 13844 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:00.564731 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.507107 12115 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:00.474651 13791 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:00.565966 12113 raft_consensus.cc:2368] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
I20260709 13:16:00.503439 13997 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:16:00.786394 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 272724us
I20260709 13:16:00.518620 13917 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:00.785944 14020 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
W20260709 13:16:00.786728 13833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.018s	sys 0.027s
W20260709 13:16:00.859071 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 667045us
I20260709 13:16:00.516462 13606 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:00.561277 14030 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:00.916898 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.785542 13877 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:00.507376 14022 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:00.929699 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 440045us
I20260709 13:16:00.805790 13935 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:00.948619 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 444005us
I20260709 13:16:01.092197 11514 leader_election.cc:400] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:01.092573 11514 leader_election.cc:403] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:00.906626 14011 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 13:16:01.089085 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:00.903755 13934 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.860451 13921 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:16:00.965011 11399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 316) took 2476 ms (client timeout 2210 ms). Trace:
W20260709 13:16:00.951613 14021 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:00.913565 14048 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:00.951334 12117 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.110951 13971 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:01.111553 13971 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.108830 13935 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.111927 13934 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.115624 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:55066 (request call id 497) took 2661 ms. Trace:
I20260709 13:16:01.116307 12117 rpcz_store.cc:276] 0709 13:15:58.460806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.483785 (+ 22979us) service_pool.cc:224] Handling call
0709 13:16:00.561719 (+2077934us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4c00113d80. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da7be5a20 00007f8da7bf6a29 00007f8da7bf713f 00007f8da7bbb2ab 00007f8da7be8f3c 00007f8da7be93fe 00007f8da7e7688b 00007f8da9875495 00007f8da98c630f 00007f8da9910a55 00007f8dae8e0cd0
0709 13:16:01.115500 (+553781us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51799936}
I20260709 13:16:01.117278 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1208) took 1300 ms. Trace:
I20260709 13:16:01.117839 11398 rpcz_store.cc:276] 0709 13:15:59.816816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.300890 (+484074us) service_pool.cc:224] Handling call
0709 13:16:00.301142 (+   252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.441001 (+139859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.441032 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.460394 (+ 19362us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.938657 (+478263us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:00.938703 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.938776 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.117066 (+178290us) raft_consensus.cc:1685] finished
0709 13:16:01.117085 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.117216 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137280,"raft.queue_time_us":138921,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"thread_start_us":435206,"threads_started":1,"wal-append.queue_time_us":631459}
W20260709 13:16:01.127768 12086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.070s	sys 0.095s
W20260709 13:16:01.128270 12086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.070s	sys 0.095s
W20260709 13:16:01.059634 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47443.4us
I20260709 13:16:01.072059 13923 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:01.114741 13934 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:00.986024 13859 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:01.085356 14041 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.581s	user 0.000s	sys 0.001s
W20260709 13:16:01.099372 11399 rpcz_store.cc:269] 0709 13:15:58.478822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.548030 (+ 69208us) service_pool.cc:224] Handling call
0709 13:16:00.955304 (+2407274us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b5400349254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:00.955517 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411842432}
W20260709 13:16:01.077884 12084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.011s	sys 0.026s
I20260709 13:16:01.132026 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:01.100337 13989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.015s	sys 0.023s
W20260709 13:16:01.138549 12084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.011s	sys 0.026s
W20260709 13:16:01.082120 13972 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.719s	user 0.007s	sys 0.009s
W20260709 13:16:01.109009 14005 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.505s	user 0.012s	sys 0.003s
I20260709 13:16:01.120679 13971 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:01.119431 14022 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:01.164242 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 376828us
I20260709 13:16:01.131582 13921 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:01.086431 13877 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.135110 13923 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:01.182333 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50379.5us
W20260709 13:16:01.160905 13989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.016s	sys 0.023s
I20260709 13:16:01.191659 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1191) took 2931 ms. Trace:
I20260709 13:16:01.193220 11393 rpcz_store.cc:276] 0709 13:15:58.260537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.467140 (+1206603us) service_pool.cc:224] Handling call
0709 13:15:59.467520 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.510540 (+ 43020us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:59.510835 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:15:59.905578 (+394743us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b18003c85c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:16:00.125892 (+220314us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b18003c85c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240
0709 13:16:00.600993 (+475101us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.601566 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.601770 (+   204us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:00.601829 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.601916 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.191347 (+589431us) raft_consensus.cc:1685] finished
0709 13:16:01.191378 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.191575 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349224,"prepare.queue_time_us":175,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":373335,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":230,"spinlock_wait_cycles":241424000,"thread_start_us":329836,"threads_started":2,"wal-append.queue_time_us":680803}
W20260709 13:16:01.197116 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 69612.2us
I20260709 13:16:01.155555 13997 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:01.210611 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:01.175297 13859 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:01.139791 14048 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:16:01.216827 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:01.131155 13818 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.199626 13498 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:01.210114 13563 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.212393 13917 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:01.196789 13827 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:01.302261 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 77167us
I20260709 13:16:01.216112 13873 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.342440 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 325) took 1355 ms. Trace:
I20260709 13:16:01.218956 13791 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.295558 13833 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.298026 12117 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:01.298275 13844 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:01.330873 14047 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:01.369822 14036 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
W20260709 13:16:01.406891 11368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.015s	sys 0.048s
I20260709 13:16:01.223224 13859 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.369565 14001 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.416488 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:01.369789 14066 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.376042 14050 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:01.369807 14043 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:01.406375 13956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.009s	sys 0.050s
I20260709 13:16:01.311177 13936 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:01.397070 13926 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.397269 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:01.422158 11397 rpcz_store.cc:276] 0709 13:15:59.987496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.461884 (+474388us) service_pool.cc:224] Handling call
0709 13:16:00.465526 (+  3642us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:00.491947 (+ 26421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.491982 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:00.506566 (+ 14584us) log.cc:844] Serialized 78 byte log entry
0709 13:16:01.097796 (+591230us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:01.097837 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.097910 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.342118 (+244208us) raft_consensus.cc:1685] finished
0709 13:16:01.342147 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.342356 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114315,"raft.queue_time_us":35551,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3746,"thread_start_us":496959,"threads_started":1,"wal-append.queue_time_us":812557}
W20260709 13:16:01.417179 14008 consensus_meta.cc:287] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.955s	user 0.010s	sys 0.002s
W20260709 13:16:01.455247 11368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.016s	sys 0.048s
I20260709 13:16:01.397070 13997 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:01.458084 14056 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:16:01.458814 14053 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.258s	user 0.000s	sys 0.000s
W20260709 13:16:01.459614 14039 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:16:01.459689 13845 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:01.460482 14059 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:16:01.461448 14074 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:01.461081 14047 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.561985 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 361) took 1186 ms. Trace:
I20260709 13:16:01.501593 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:01.563112 14030 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.562383 11476 rpcz_store.cc:276] 0709 13:16:00.375320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.375520 (+   200us) service_pool.cc:224] Handling call
0709 13:16:00.375908 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.376214 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.376240 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.402619 (+ 26379us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.570413 (+167794us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:00.570469 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.570565 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.561779 (+991214us) raft_consensus.cc:1685] finished
0709 13:16:01.561803 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.561930 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6942,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":22182,"thread_start_us":102632,"threads_started":1,"wal-append.queue_time_us":693579}
I20260709 13:16:01.563477 14047 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.563652 14030 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.564165 14030 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.416198 13498 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.566428 14030 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.566900 13498 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.568581 13859 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.569118 13859 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.569509 13859 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.465767 14006 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:01.422487 13917 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.438017 14040 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:01.424458 13827 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.581092 13827 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.471707 13871 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.582155 13867 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:01.485563 14066 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:01.475056 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 58647.2us
I20260709 13:16:01.355409 13921 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.450661 13791 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.415926 13563 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:01.490757 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 398702us
I20260709 13:16:01.482257 13159 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:01.479242 13844 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.437799 13873 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.602735 11542 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:01.603909 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:01.604754 11545 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:16:01.482581 14050 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:01.512425 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.513288 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1715) took 2601 ms. Trace:
I20260709 13:16:01.547931 14081 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:01.559049 13845 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:01.559433 13983 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.560149 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1210) took 1574 ms. Trace:
I20260709 13:16:01.573422 12123 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:01.490687 14043 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.576026 14006 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:01.484302 14001 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.577878 13917 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.578505 14040 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.497916 13950 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:01.589190 11551 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:01.590163 11546 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:01.596172 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:01.671871 11550 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.567342 14064 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.609517 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:01.629488 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:01.633760 11543 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:01.629006 11395 rpcz_store.cc:276] 0709 13:15:59.985389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.442398 (+457009us) service_pool.cc:224] Handling call
0709 13:16:00.442778 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.443191 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.443227 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.460303 (+ 17076us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.954887 (+494584us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:00.954928 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.955014 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.559924 (+604910us) raft_consensus.cc:1685] finished
0709 13:16:01.559942 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.560087 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20028,"raft.queue_time_us":27026,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":13299,"thread_start_us":450825,"threads_started":1,"wal-append.queue_time_us":698433}
I20260709 13:16:01.645274 11473 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:01.607537 11549 rpcz_store.cc:276] 0709 13:15:58.912086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.179329 (+267243us) service_pool.cc:224] Handling call
0709 13:15:59.179672 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.183748 (+  4076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.183774 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:59.205062 (+ 21288us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.993557 (+1788495us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b1800214780. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1
0709 13:16:01.048554 (+ 54997us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:01.135299 (+ 86745us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.494344 (+359045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.494453 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.494491 (+    38us) raft_consensus.cc:1685] finished
0709 13:16:01.511348 (+ 16857us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.513096 (+  1748us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204358,"prepare.queue_time_us":116263,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3428,"raft.queue_time_us":60146,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":56015,"spinlock_wait_cycles":19922688,"thread_start_us":461698,"threads_started":3,"wal-append.queue_time_us":53209,"wal-append.run_cpu_time_us":1749,"wal-append.run_wall_time_us":1078541}
I20260709 13:16:01.498332 13926 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.584098 14074 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.667095 13859 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.629791 14050 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.699304 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:01.700383 14040 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:01.676694 14030 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:01.584108 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62690.1us
I20260709 13:16:01.582156 13871 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.694720 11544 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:01.700918 14040 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.668419 13971 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.601832 13791 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.658869 14003 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.701428 14030 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.589313 13827 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:16:01.597960 14065 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:01.709317 13802 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:01.722131 11550 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:01.683624 13758 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.631541 13844 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.711648 14012 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.738125 13859 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:00.603505 11401 rpcz_store.cc:276] 0709 13:15:58.273428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.480330 (+1206902us) service_pool.cc:224] Handling call
0709 13:15:59.527109 (+ 46779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.527374 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:59.818923 (+291549us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.367271 (+548348us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b18003c8aa0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8da682ea40 00007f8da68294e4 00007f8da6829455 00007f8da9911060 00007f8da9923716 00007f8da9934287 00007f8da9934208
0709 13:16:00.492497 (+125226us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.519720 (+ 27223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:00.519776 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:00.519826 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.519917 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.520046 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199986,"prepare.queue_time_us":397222,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":81,"raft.queue_time_us":421412,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":53505,"spinlock_wait_cycles":45449600,"thread_start_us":586140,"threads_started":3,"wal-append.queue_time_us":352444}
I20260709 13:16:01.705669 13917 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:01.765589 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:01.766505 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:01.701459 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:01.718125 14077 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.752427 12114 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:01.767817 12113 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:01.752139 14072 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.755484 13859 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.761421 14065 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.763814 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.644119 14081 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.764839 11474 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:01.764935 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:01.743557 11469 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:01.643852 13983 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.774830 11472 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:16:01.764688 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.781739 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.782056 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.782541 11469 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:01.735677 14058 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.743382 14067 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.737653 14070 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:01.789116 11468 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:01.790678 11548 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:01.790877 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.794088 11397 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:01.845278 11395 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:01.795490 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:01.846382 11399 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:01.846726 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.847673 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:01.796373 11393 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:01.848284 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.849076 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:01.849364 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.850003 11395 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:01.850803 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.851529 11395 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:01.800755 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.800983 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.711386 14040 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.801430 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.848316 11399 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.738710 13899 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:16:01.852500 11472 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:16:01.860805 11469 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:01.862066 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:01.852530 11474 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 3.
I20260709 13:16:01.799767 14048 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:01.801656 14047 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.743335 13802 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.751163 13758 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:01.860086 13923 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
I20260709 13:16:01.704010 14064 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.729131 14006 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.767921 14077 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:01.860805 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75586.9us
I20260709 13:16:01.933672 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:01.733644 14003 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.840981 14069 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.921118 14030 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:01.921350 14093 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:01.783927 13917 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.922000 14040 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.905526 14057 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:16:01.931532 14074 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:01.776568 14012 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.775496 14072 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:01.947849 14071 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.347s	user 0.004s	sys 0.004s
I20260709 13:16:01.905712 14048 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.956115 14072 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.956910 14048 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.959753 13791 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.804167 14058 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:01.936205 14089 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:01.905543 14047 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.960625 13844 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:01.967592 14076 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.324s	user 0.001s	sys 0.001s
I20260709 13:16:01.968813 14081 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.843489 14070 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.992645 14003 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:01.974794 13756 consensus_meta.cc:287] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.590s	user 0.010s	sys 0.003s
I20260709 13:16:01.963979 14066 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.841696 14067 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.979043 14084 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:16:01.948396 14040 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:16:02.005348 13971 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.009158 14077 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.923086 13923 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.025063 13923 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.978579 13926 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:01.953835 14057 consensus_queue.cc:1243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:01.949378 13554 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.030471 13554 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:01.990087 14044 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.083s	user 0.003s	sys 0.003s
I20260709 13:16:01.990880 13983 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.981333 14089 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:01.996304 14085 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:16:01.997184 13802 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.000967 13756 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:02.001732 14067 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:01.995134 14001 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.004033 13917 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.004350 13859 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:01.986008 14065 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.002746 14064 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.068756 14089 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.070866 13954 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:02.071486 13954 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:02.072700 14070 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.003157 13758 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.072541 13954 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.076748 13954 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:02.076515 14050 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.075502 14047 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.086840 14050 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.091418 13833 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:02.092011 13833 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:02.093029 13833 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.095026 14010 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:02.095551 14010 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:02.096586 14010 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:02.100664 14042 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.182s	user 0.000s	sys 0.003s
I20260709 13:16:02.112866 13999 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.132560 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:02.135383 14010 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:02.133172 14012 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.119647 13606 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.139288 13606 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:02.149842 13554 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.155508 13563 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.159031 14043 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.160913 13871 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.166338 13818 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:02.177853 14038 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.390s	user 0.004s	sys 0.000s
I20260709 13:16:02.168916 13873 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.172914 14006 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:02.181661 13833 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.185154 14058 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:02.237586 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:02.245682 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.245898 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.248498 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 307) took 4170 ms. Trace:
I20260709 13:16:02.249097 11394 rpcz_store.cc:276] 0709 13:15:58.077459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.241973 (+1164514us) service_pool.cc:224] Handling call
0709 13:15:59.242286 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.294621 (+1052335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.247849 (+1953228us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400230854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:02.248291 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401909504}
I20260709 13:16:02.251119 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:02.252439 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.253059 11395 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:02.262424 11442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.703s (SENT) [suppressed 14 similar messages]
I20260709 13:16:02.263118 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.263522 11393 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:02.263823 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.264820 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:01.913923 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:02.266428 11393 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:02.267292 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.253942 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.267463 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1641) took 3374 ms. Trace:
I20260709 13:16:02.267513 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.268358 11393 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:02.268787 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.269644 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.254782 11401 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:02.259651 11398 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:16:02.270720 11397 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:02.276533 11473 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:02.268005 11400 rpcz_store.cc:276] 0709 13:15:58.893276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.506268 (+612992us) service_pool.cc:224] Handling call
0709 13:15:59.506584 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.565277 (+ 58693us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:59.968512 (+403235us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.385696 (+417184us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.441188 (+ 55492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:00.441230 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:00.441266 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.267041 (+1825775us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b5400231254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:02.267097 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:02.267300 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269205,"prepare.queue_time_us":533244,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":172,"raft.queue_time_us":318442,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":19694,"spinlock_wait_cycles":134177152,"thread_start_us":584489,"threads_started":2,"wal-append.queue_time_us":229865,"wal-append.run_cpu_time_us":2989,"wal-append.run_wall_time_us":1587947}
W20260709 13:16:02.276746 11470 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 347) took 2985 ms (client timeout 1876 ms). Trace:
W20260709 13:16:02.277211 11470 rpcz_store.cc:269] 0709 13:15:59.291376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.328006 (+ 36630us) service_pool.cc:224] Handling call
0709 13:16:02.276256 (+2948250us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b540028bc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:02.276516 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625503488}
I20260709 13:16:02.277545 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.278363 11473 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:02.278751 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.280390 11470 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:02.281399 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.281556 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.282374 11399 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:02.283511 11470 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:02.265458 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.285606 11469 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
W20260709 13:16:02.286252 11299 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 141) took 12752 ms (client timeout 15000 ms). Trace:
I20260709 13:16:02.287272 11469 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:02.289184 11469 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:02.253527 11396 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:16:02.291232 11396 raft_consensus.cc:614] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:02.279625 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.294148 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41722 (request call id 80) took 3197 ms. Trace:
I20260709 13:16:02.260965 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.295084 11394 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:02.294632 11473 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:16:02.300599 11472 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:16:02.261339 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.303478 11393 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:02.304335 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.294615 11396 rpcz_store.cc:276] 0709 13:15:59.096640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.538936 (+442296us) service_pool.cc:224] Handling call
0709 13:16:02.293842 (+2754906us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b54003a6bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9913853 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:02.294012 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6220591872}
W20260709 13:16:02.286789 11299 rpcz_store.cc:269] 0709 13:15:49.533522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.555193 (+ 21671us) service_pool.cc:224] Handling call
0709 13:15:53.175718 (+3620525us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b5400018028. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dad283161 00007f8dad30d3af 00007f8dad30eb11 00007f8daf80f2d1 00007f8daf80fb2f 00007f8daf6ced4b 00007f8daf7d6c3e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc
0709 13:16:02.285978 (+9110260us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:53.304805 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:53.305499 (+   694us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:53.305546 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:15:53.305559 (+    13us) tablet.cc:662] Decoding operations
0709 13:15:53.317074 (+ 11515us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:53.317152 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:53.317191 (+    39us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:15:53.318931 (+  1740us) tablet.cc:701] Row locks acquired
0709 13:15:53.318980 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:15:53.383100 (+ 64120us) write_op.cc:270] Start()
0709 13:15:53.383267 (+   167us) write_op.cc:276] Timestamp: P: 1783602953369833 usec, L: 0
0709 13:15:53.383290 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:53.490107 (+106817us) log.cc:844] Serialized 9276 byte log entry
0709 13:15:53.821748 (+331641us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da680f0ac 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b
0709 13:15:54.307806 (+486058us) op_driver.cc:464] REPLICATION: finished
0709 13:15:54.460865 (+153059us) write_op.cc:301] APPLY: starting
0709 13:15:54.460967 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:54.461544 (+   577us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:54.461583 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:54.464412 (+  2829us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:54.464824 (+   412us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:15:54.464877 (+    53us) write_op.cc:312] APPLY: finished
0709 13:15:54.675460 (+210583us) log.cc:844] Serialized 164 byte log entry
0709 13:15:54.733952 (+ 58492us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:54.770415 (+ 36463us) write_op.cc:454] Released schema lock
0709 13:15:54.801151 (+ 30736us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:55.004509 (+203358us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:55.012697 (+  8188us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13423,"reported_tablets":27,"spinlock_wait_cycles":11560192,"child_traces":[["op",{"apply.queue_time_us":140068,"apply.run_cpu_time_us":8418,"apply.run_wall_time_us":418556,"num_ops":24,"prepare.queue_time_us":127920,"prepare.run_cpu_time_us":24110,"prepare.run_wall_time_us":557370,"raft.queue_time_us":65205,"raft.run_cpu_time_us":10479,"raft.run_wall_time_us":200451,"replication_time_us":907013,"spinlock_wait_cycles":7120128,"thread_start_us":410312,"threads_started":4,"wal-append.queue_time_us":397410,"wal-append.run_cpu_time_us":26700,"wal-append.run_wall_time_us":1941218}]]}
W20260709 13:16:02.306411 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.314535 11402 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:16:02.317085 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 383300us
I20260709 13:16:02.318773 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:02.324777 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:02.326535 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:02.332134 11365 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.333845 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.283891 11475 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:02.265295 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.385457 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.386442 11543 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:16:02.387426 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.387619 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.389147 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.390142 11546 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:02.390367 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.391594 11545 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:02.389257 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.396104 13756 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.396953 13756 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.398312 13554 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.399327 13563 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.399364 13763 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.400044 13563 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.400178 13763 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.400768 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:02.405864 13563 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.406720 13763 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.406888 13758 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:02.407496 13763 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.407750 13758 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.408320 13763 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.410415 13758 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.410581 13763 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.313944 11297 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.414177 11297 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.415974 11297 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
W20260709 13:16:02.417284 12123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 421) took 2035 ms (client timeout 1914 ms). Trace:
W20260709 13:16:02.418764 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1642) took 3524 ms (client timeout 1709 ms). Trace:
W20260709 13:16:02.420593 11402 rpcz_store.cc:269] 0709 13:15:58.894367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.923229 (+ 28862us) service_pool.cc:224] Handling call
0709 13:16:02.418503 (+3495274us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b54003a6bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:02.418651 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7629091200}
I20260709 13:16:02.435719 11546 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:02.435170 11297 catalog_manager.cc:5697] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 3, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:02.437579 13756 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.385481 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:02.420284 12123 rpcz_store.cc:269] 0709 13:16:00.381852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.382003 (+   151us) service_pool.cc:224] Handling call
0709 13:16:02.416833 (+2034830us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54005cec54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:02.417013 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653016960}
I20260709 13:16:02.452447 11551 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:16:02.473398 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.437669 11297 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.471987 11395 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:02.463788 11551 pending_rounds.cc:85] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 3
I20260709 13:16:02.486899 11472 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:02.487116 13755 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.487902 13755 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.490257 13755 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.490856 13606 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:02.491283 13867 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:02.492524 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.491356 13606 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.493885 13990 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:02.494521 13990 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.495059 13990 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.437722 13917 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:02.498450 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 97937.5us
I20260709 13:16:02.499426 13767 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:02.500666 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 174330us
I20260709 13:16:02.501505 13763 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:02.503127 13680 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.512585 13763 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.513123 13763 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:02.507223 13873 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:02.512943 13680 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.503667 13641 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.496452 13606 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:02.513783 13873 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.514504 13873 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:02.491886 13867 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.516870 13867 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:02.432319 13758 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.513175 11475 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:02.483384 11551 pending_rounds.cc:107] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:16:02.527117 11551 raft_consensus.cc:2889] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:02.513617 11297 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.535987 11297 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.537611 11297 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.539342 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:02.536011 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.544448 11543 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:16:02.561034 13950 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.563476 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.563745 13950 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.564235 11548 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:02.565229 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.530514 13767 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.566268 13767 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.485492 13954 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.572454 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:02.573289 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:02.485632 13554 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.574728 13934 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.540414 14006 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:02.575525 13934 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.576129 14006 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.577329 11515 leader_election.cc:400] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:02.576968 11297 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.578598 14006 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:02.586477 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.599398 11548 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:02.600046 14070 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.600291 14010 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:02.600757 14010 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.601194 14010 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:02.600869 14070 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:02.603411 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 269780us
I20260709 13:16:02.604427 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.599977 13767 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.611229 13935 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.610865 13758 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.611866 13935 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.613464 13758 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.614147 13935 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.610965 13950 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.615041 14067 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.615990 14067 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.563800 13917 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.619940 13917 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:02.621272 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.609762 11543 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:02.626322 14027 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.626607 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.629118 11551 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:02.626765 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.651909 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.528208 13641 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.669204 13641 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.670585 11544 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:16:02.672413 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.673760 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.676841 13936 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:02.677561 13936 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.679983 13934 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.579280 11297 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 2 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
W20260709 13:16:02.653784 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.629009 14027 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.655246 13802 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:02.614166 14002 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:02.680434 13755 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:02.741119 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.682080 13936 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.674849 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.666415 13606 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.656632 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.652594 11551 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:16:02.685268 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:02.685576 11348 auto_rebalancer.cc:893] Removing replace marker failed: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:02.686179 13680 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.687738 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.688212 14067 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.689252 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:02.692373 14070 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:02.692900 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 69205.2us
W20260709 13:16:02.693348 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 114458us
I20260709 13:16:02.699218 14065 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:02.756779 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67734.7us
I20260709 13:16:02.699322 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.702782 13554 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.705097 13873 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:02.705948 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.614835 13935 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:02.716625 11395 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:02.717628 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.718147 11396 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:02.718739 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:02.718884 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.784312 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.720289 11546 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:02.721002 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.724485 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.724795 14010 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.727943 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.728045 13990 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.791014 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.729022 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:02.730741 13867 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.796754 11472 raft_consensus.cc:2393] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:02.740387 11550 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:02.751261 11348 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:16:02.752447 11545 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:16:02.760509 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.798825 11394 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:02.761235 13936 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:02.761677 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.764475 13606 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.765872 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.766723 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.808314 11544 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
I20260709 13:16:02.768678 14065 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.770358 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.740253 13802 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.773512 14070 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:02.809700 11400 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:02.790037 11473 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:02.787571 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.794239 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.794239 13767 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:02.795428 11515 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:02.795786 11393 raft_consensus.cc:2368] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
W20260709 13:16:02.803261 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.780335 13935 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:02.837570 13935 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:02.837731 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.779496 11297 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:02.839200 11475 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:16:02.797787 11348 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:16:02.769511 14002 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.785961 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.807790 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:02.752856 13763 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.815557 11470 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:02.814831 11543 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:02.829056 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.809417 13936 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:02.835088 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:02.846478 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.836088 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:02.847239 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:02.754200 13641 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:16:02.847904 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:02.850754 11397 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:16:02.900733 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54357.6us
I20260709 13:16:02.906461 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:02.850838 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 85099.1us
I20260709 13:16:02.860214 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.849424 11395 raft_consensus.cc:2376] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7. Re-sending same reply.
I20260709 13:16:02.894556 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:02.848904 11394 raft_consensus.cc:2376] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5. Re-sending same reply.
I20260709 13:16:02.900904 11551 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.851745 11545 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:02.829602 13802 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:03.081698 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:02.912441 13680 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:03.083068 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:02.911136 13877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.081s	sys 0.095s
I20260709 13:16:03.084704 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:03.085711 13833 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:03.086110 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:03.086133 13833 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.086733 13833 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.907809 14096 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:03.028738 13873 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.894860 14070 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.094056 14070 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:02.845443 13755 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:02.911346 14052 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.095795 14052 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:02.976362 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:03.030138 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 91284.7us
I20260709 13:16:02.943158 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.097206 11297 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 3 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:03.065433 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:03.082876 13859 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:03.083356 13680 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:03.084227 14064 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:03.084935 13877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.514s	user 0.084s	sys 0.116s
W20260709 13:16:02.907898 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 100186us
I20260709 13:16:03.071520 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:02.920209 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.084224 11542 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:03.066476 11399 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:03.068569 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:03.071223 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 282880us
I20260709 13:16:03.102835 13859 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.105909 11400 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:02.852407 13990 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:02.850958 13936 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:02.907496 13828 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:03.115751 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50409.3us
W20260709 13:16:02.934439 13956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.088s	sys 0.127s
I20260709 13:16:02.976711 13934 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:03.141657 11468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.017s	sys 0.030s
I20260709 13:16:03.141927 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.096787 13755 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:02.850560 14065 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:03.123175 11393 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.975891 13867 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:03.067174 13763 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:03.122004 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:03.150480 11515 leader_election.cc:400] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
I20260709 13:16:03.123759 13859 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:03.115038 14064 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.157203 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.108597 13680 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:03.142325 11468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.017s	sys 0.030s
I20260709 13:16:02.984230 13554 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:03.139627 13828 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.145812 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.144987 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:03.150923 11472 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.150938 11515 leader_election.cc:403] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:16:03.130376 11550 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:03.106065 14090 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:03.108161 13873 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.155041 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.158845 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:03.179553 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.906512 14027 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:03.136860 13990 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:03.159802 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 73788.4us
W20260709 13:16:03.185514 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.148036 13956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.092s	sys 0.132s
W20260709 13:16:03.163476 11474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.074s	sys 0.110s
W20260709 13:16:03.207302 11474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.074s	sys 0.110s
I20260709 13:16:03.207101 11399 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:03.183195 11400 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:03.156472 11297 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 4 to 2, term changed from 4 to 3, leader changed from <none> to 147201d22a694ca59f6e014f083a0892 (127.10.3.1), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from VOTER to NON_VOTER. New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:16:03.161688 14064 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.167588 13828 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:03.201025 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.163564 14030 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.208915 13606 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
W20260709 13:16:03.224594 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45908us
I20260709 13:16:03.221741 11297 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
W20260709 13:16:03.250893 11477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.005s	sys 0.041s
W20260709 13:16:03.251098 12083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.320s	user 0.017s	sys 0.038s
W20260709 13:16:03.251406 11477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.348s	user 0.005s	sys 0.041s
W20260709 13:16:03.251559 12083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.017s	sys 0.038s
W20260709 13:16:03.251958 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 1.40875e+06us
I20260709 13:16:03.186723 14010 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:03.160852 13934 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.255324 11401 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:03.188329 13990 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.163782 13802 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.253744 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1158) took 1454 ms. Trace:
W20260709 13:16:03.274412 11476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.532s	user 0.037s	sys 0.040s
I20260709 13:16:03.274938 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:03.279548 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97078.7us
W20260709 13:16:03.325572 11476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.679s	user 0.037s	sys 0.041s
I20260709 13:16:03.208828 13554 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.221817 13873 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:03.408001 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 273899us
I20260709 13:16:03.361375 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:03.345067 11548 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:03.409469 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63543.7us
W20260709 13:16:03.397190 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.414326 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 130373us
I20260709 13:16:03.414696 11399 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:16:03.407161 11471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.327s	user 0.028s	sys 0.025s
W20260709 13:16:03.407312 11547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.524s	user 0.010s	sys 0.030s
I20260709 13:16:03.418593 11393 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:03.279026 13990 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.427874 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:03.428022 14099 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 13:16:03.333940 11297 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
W20260709 13:16:03.406980 13921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.015s	sys 0.025s
W20260709 13:16:03.444558 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.602s	user 0.028s	sys 0.026s
W20260709 13:16:03.445709 11547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.738s	user 0.011s	sys 0.031s
I20260709 13:16:03.451304 14099 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.455914 11550 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:03.457089 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.457752 11550 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:03.344516 13755 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:03.373907 13997 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.725s	user 0.015s	sys 0.041s
I20260709 13:16:03.326864 13606 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:16:03.422775 14002 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:03.502748 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 65289us
I20260709 13:16:03.435169 13818 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:03.498201 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 165602us
I20260709 13:16:03.500110 11297 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.407568 13833 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:03.459151 13921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.527s	user 0.016s	sys 0.025s
I20260709 13:16:03.506342 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:03.505179 13997 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.854s	user 0.019s	sys 0.042s
I20260709 13:16:03.491569 14065 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:03.519111 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:03.505681 13984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.326s	user 0.010s	sys 0.033s
I20260709 13:16:03.532783 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:03.509519 13818 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.524185 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 364) took 1914 ms. Trace:
I20260709 13:16:03.535624 11367 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:03.536178 11297 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.511198 13606 consensus_queue.cc:1243] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:03.535380 13802 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:03.545761 11550 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:16:03.543450 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.617s	user 0.021s	sys 0.030s
I20260709 13:16:03.551275 11542 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:03.545527 11297 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:03.518318 14052 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:03.555639 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.828s	user 0.021s	sys 0.031s
I20260709 13:16:03.539932 13818 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:03.543499 13984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.626s	user 0.010s	sys 0.033s
I20260709 13:16:03.541222 14027 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:03.536958 11476 rpcz_store.cc:276] 0709 13:16:01.602027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.602603 (+   576us) service_pool.cc:224] Handling call
0709 13:16:01.643444 (+ 40841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.643646 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:01.646482 (+  2836us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.414409 (+1767927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:03.445191 (+ 30782us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:03.494109 (+ 48918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.508820 (+ 14711us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.516313 (+  7493us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21764,"prepare.queue_time_us":3300,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":26863,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":25778,"thread_start_us":1679303,"threads_started":1}
I20260709 13:16:03.576442 11551 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:03.579563 11297 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 4, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 4 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:03.583098 11516 leader_election.cc:400] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:03.583565 11516 leader_election.cc:403] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:03.584470 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.535745 13755 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.581100 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1144) took 1838 ms. Trace:
W20260709 13:16:03.565593 13935 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.003s
I20260709 13:16:03.613574 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.554169 13758 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:03.675444 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 112101us
I20260709 13:16:03.670424 11297 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 2, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 2 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:03.553481 14065 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.692276 11297 catalog_manager.cc:5697] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 3, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.615123 11471 rpcz_store.cc:276] 0709 13:16:01.762150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.762315 (+   165us) service_pool.cc:224] Handling call
0709 13:16:01.762619 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.762816 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:01.842070 (+ 79254us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.517885 (+1675815us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:03.539111 (+ 21226us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:03.547737 (+  8626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.555520 (+  7783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.581041 (+ 25521us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36415,"prepare.queue_time_us":17339,"prepare.run_cpu_time_us":587,"prepare.run_wall_time_us":35317,"thread_start_us":1681669,"threads_started":2,"wal-append.queue_time_us":1732402}
I20260709 13:16:03.617564 11551 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:03.627974 13833 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:03.629519 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.567060 13752 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.640048 13984 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:03.644389 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.646759 14003 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:03.650934 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 365) took 2022 ms. Trace:
I20260709 13:16:03.703696 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:03.653142 14090 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.660997 14030 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:03.664227 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1755) took 1148 ms. Trace:
I20260709 13:16:03.278352 11474 rpcz_store.cc:276] 0709 13:16:01.768105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.292129 (+524024us) service_pool.cc:224] Handling call
0709 13:16:02.292556 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.292981 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:02.293438 (+   457us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.219779 (+926341us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:03.219859 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:03.219920 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.220004 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.222439 (+  2435us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6972,"prepare.queue_time_us":90895,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":65,"raft.queue_time_us":12229,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":31635,"thread_start_us":913981,"threads_started":1}
W20260709 13:16:03.587368 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 1.85969e+06us
W20260709 13:16:03.540530 11469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.040s	sys 0.059s
W20260709 13:16:03.707020 11469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.373s	user 0.040s	sys 0.059s
I20260709 13:16:03.673516 13758 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.706537 11548 rpcz_store.cc:276] 0709 13:16:02.515966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.839472 (+323506us) service_pool.cc:224] Handling call
0709 13:16:02.858097 (+ 18625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.415838 (+557741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.663824 (+247986us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400404554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:03.664106 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091239168}
I20260709 13:16:03.711561 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1157) took 1944 ms. Trace:
I20260709 13:16:03.712255 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:03.695684 13802 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.718053 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.718853 11550 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:03.716559 11545 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:03.672487 13935 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.552101 14099 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:03.707268 14090 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:03.724450 13879 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:03.735741 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.736781 11548 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:03.704071 11477 rpcz_store.cc:276] 0709 13:16:01.628816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.630091 (+  1275us) service_pool.cc:224] Handling call
0709 13:16:01.630465 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.844042 (+213577us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:01.902886 (+ 58844us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.252009 (+1349123us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:03.252103 (+    94us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:03.252170 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.650570 (+398400us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b540044acd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:03.650635 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.650850 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":61567,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":18780,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":10035,"spinlock_wait_cycles":490599808,"thread_start_us":1348185,"threads_started":1}
I20260709 13:16:03.712056 11469 rpcz_store.cc:276] 0709 13:16:01.767525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.290027 (+522502us) service_pool.cc:224] Handling call
0709 13:16:02.290474 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.290931 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:02.333858 (+ 42927us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.711134 (+1377276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:03.711199 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:03.711265 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.711352 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.711496 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86263,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":335,"thread_start_us":1373369,"threads_started":1}
W20260709 13:16:03.772595 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.694700 11297 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 4, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 4 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:03.713402 13934 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:03.832253 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.833058 11393 raft_consensus.cc:2368] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:03.837306 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.724071 13758 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:03.732475 13873 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:03.708366 14030 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.859650 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:03.854008 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.854405 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.836230 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:03.856053 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.749547 13879 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:03.840952 13606 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.896389 13606 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.800618 11297 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.827780 13763 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:03.858662 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.892678 11402 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:03.854743 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:03.835664 13554 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:03.881840 11516 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:03.911605 11548 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:03.917892 11297 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.911929 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:03.849570 14097 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:16:03.877216 13936 consensus_meta.cc:287] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
W20260709 13:16:03.873292 14064 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.600s	user 0.006s	sys 0.005s
W20260709 13:16:03.858961 13680 consensus_meta.cc:287] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.001s
I20260709 13:16:03.922268 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:03.922690 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.853549 13859 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 13:16:03.928573 13879 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:03.930423 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 316098us
W20260709 13:16:03.947613 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.947073 13984 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:16:03.932022 11297 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 3 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:03.874567 13997 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:03.915727 13828 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.003s
I20260709 13:16:03.839272 14099 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.958940 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:03.944029 13859 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.938515 13936 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.942888 13680 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:03.939706 14064 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:03.977247 11473 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:03.994769 13984 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:16:03.996867 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44441.3us
I20260709 13:16:03.998919 13554 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:03.999538 13554 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:04.000485 13554 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.010932 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:04.006837 11472 consensus_meta.cc:287] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.825s	user 0.009s	sys 0.000s
I20260709 13:16:04.013752 11476 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:04.013486 11472 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:03.972715 13763 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.028752 13763 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:04.029338 13763 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.029790 13763 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.010319 13755 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:03.961233 13828 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:04.036773 13828 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.039518 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:04.042727 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1782) took 1182 ms. Trace:
I20260709 13:16:03.998745 14027 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:04.034770 14099 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.056428 11476 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:04.044302 11472 rpcz_store.cc:276] 0709 13:16:02.833748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.895479 (+ 61731us) service_pool.cc:224] Handling call
0709 13:16:03.121813 (+226334us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:04.015924 (+894111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31129472}
I20260709 13:16:04.064057 11402 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.048974 14065 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.047523 13935 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.013048 13997 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.098898 13859 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.088416 14030 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.060736 13984 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:16:04.088119 14027 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:04.103513 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1150) took 2337 ms. Trace:
W20260709 13:16:04.095288 13990 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.636s	user 0.006s	sys 0.006s
W20260709 13:16:04.110936 11514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.212s	user 0.033s	sys 0.050s
I20260709 13:16:04.091820 13763 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:04.114262 13680 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:04.117705 12085 connection.cc:256] RPC call timeout handler was delayed by -2.1726200470000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
I20260709 13:16:04.107558 11468 rpcz_store.cc:276] 0709 13:16:01.765757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.282511 (+516754us) service_pool.cc:224] Handling call
0709 13:16:02.283194 (+   683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:02.283831 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.283865 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:02.291073 (+  7208us) log.cc:844] Serialized 26 byte log entry
0709 13:16:03.158593 (+867520us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:03.158644 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.158750 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.103237 (+944487us) raft_consensus.cc:1685] finished
0709 13:16:04.103264 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.103418 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162885,"raft.queue_time_us":200310,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":4758,"thread_start_us":867045,"threads_started":1,"wal-append.queue_time_us":1807897}
W20260709 13:16:04.125743 11514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.033s	sys 0.050s
I20260709 13:16:04.132602 13802 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:04.132889 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:04.105990 13997 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.110812 14027 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.164611 13606 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:04.164812 13989 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.013s	user 0.032s	sys 0.090s
I20260709 13:16:04.164844 14027 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:04.165395 13989 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.034s	user 0.032s	sys 0.090s
I20260709 13:16:04.165437 13606 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:04.166615 11549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.620s	user 0.029s	sys 0.060s
W20260709 13:16:04.167054 11549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.655s	user 0.030s	sys 0.060s
W20260709 13:16:04.139499 11544 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.164s	user 0.022s	sys 0.054s
W20260709 13:16:04.170034 11544 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.194s	user 0.023s	sys 0.055s
I20260709 13:16:04.157142 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 287) took 2549 ms. Trace:
W20260709 13:16:04.164403 12086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.210s	user 0.014s	sys 0.078s
I20260709 13:16:04.158164 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:04.173276 11470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.264s	user 0.026s	sys 0.064s
W20260709 13:16:04.173806 11470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.264s	user 0.026s	sys 0.064s
I20260709 13:16:04.120920 13990 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.144362 13936 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.166317 13606 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:04.170712 11547 rpcz_store.cc:276] 0709 13:16:01.602148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.602324 (+   176us) service_pool.cc:224] Handling call
0709 13:16:01.602622 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:01.603101 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:01.603143 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:01.643768 (+ 40625us) log.cc:844] Serialized 26 byte log entry
0709 13:16:03.554756 (+1910988us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:03.561628 (+  6872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.578687 (+ 17059us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.146744 (+568057us) raft_consensus.cc:1685] finished
0709 13:16:04.146778 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.151341 (+  4563us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":226,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":21766,"thread_start_us":1738519,"threads_started":1,"wal-append.queue_time_us":2458437}
W20260709 13:16:04.178335 12086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.248s	user 0.015s	sys 0.079s
I20260709 13:16:04.197249 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:04.170181 13763 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:04.201175 11395 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:16:04.200798 11475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.034s	sys 0.042s
W20260709 13:16:04.202742 11475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.035s	sys 0.042s
I20260709 13:16:04.169858 14099 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:04.219122 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86299.3us
W20260709 13:16:04.182405 13563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.035s	sys 0.042s
I20260709 13:16:04.241612 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:04.207533 13763 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:04.237931 13563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.300s	user 0.038s	sys 0.042s
W20260709 13:16:04.237628 13902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.523s	user 0.044s	sys 0.085s
I20260709 13:16:04.206817 13802 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:16:04.256455 13763 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:04.269083 13902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.710s	user 0.044s	sys 0.086s
W20260709 13:16:04.288131 14110 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:16:04.283087 13818 consensus_meta.cc:287] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.005s
I20260709 13:16:04.264074 13758 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:04.258848 14099 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.309531 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:04.317970 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:04.312655 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:04.318898 11551 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:04.320493 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:04.321422 11548 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:04.324424 11545 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:04.325857 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:04.296972 13802 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:16:04.333662 11547 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:16:04.305399 13818 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.344348 13984 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:04.362380 11398 raft_consensus.cc:2368] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:04.348649 13802 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:16:04.384994 14103 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:16:04.359151 13990 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.388082 13802 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:04.391785 13802 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:04.387311 13984 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:04.388302 13950 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.477s	user 0.027s	sys 0.074s
I20260709 13:16:04.430951 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1786) took 1259 ms. Trace:
I20260709 13:16:04.420547 13606 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:04.420347 13818 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.434700 11473 rpcz_store.cc:276] 0709 13:16:03.171124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.247029 (+ 75905us) service_pool.cc:224] Handling call
0709 13:16:04.430732 (+1183703us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b540005a854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:04.430886 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600525696}
W20260709 13:16:04.458775 13950 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.547s	user 0.027s	sys 0.075s
I20260709 13:16:04.470974 13606 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:16:04.486125 11396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.015s	sys 0.008s
W20260709 13:16:04.486491 11396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.015s	sys 0.008s
I20260709 13:16:04.472000 13758 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:04.484433 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.007s	sys 0.018s
W20260709 13:16:04.495308 11394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.009s	sys 0.002s
W20260709 13:16:04.496287 11394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.010s	sys 0.003s
W20260709 13:16:04.499312 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.008s	sys 0.018s
W20260709 13:16:04.499848 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 1.07333e+06us
I20260709 13:16:04.485807 13606 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:04.523000 13767 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.005s	sys 0.029s
I20260709 13:16:04.543468 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1788) took 1366 ms. Trace:
I20260709 13:16:04.546680 11476 rpcz_store.cc:276] 0709 13:16:03.177248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.696133 (+518885us) service_pool.cc:224] Handling call
0709 13:16:04.543287 (+847154us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400213754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:04.543406 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675144576}
W20260709 13:16:04.545739 13767 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.388s	user 0.005s	sys 0.029s
I20260709 13:16:04.550853 14100 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:04.551496 14100 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:04.552477 13984 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:04.552739 14100 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:04.555727 14113 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.288s	user 0.005s	sys 0.002s
W20260709 13:16:04.565649 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.573216 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:04.544812 13752 consensus_meta.cc:287] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.833s	user 0.013s	sys 0.000s
W20260709 13:16:04.548871 14090 consensus_meta.cc:287] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.002s
I20260709 13:16:04.606194 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:04.644841 12083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.215s	user 0.029s	sys 0.047s
W20260709 13:16:04.646106 12083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.029s	sys 0.047s
W20260709 13:16:04.646373 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 1.2165e+06us
W20260709 13:16:04.646767 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 111189us
W20260709 13:16:04.648567 11471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.008s	sys 0.023s
W20260709 13:16:04.648969 11471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.009s	sys 0.023s
I20260709 13:16:04.608824 13752 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:04.657321 13956 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.226s	user 0.005s	sys 0.027s
W20260709 13:16:04.711580 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54807.1us
W20260709 13:16:04.711508 11517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.008s	sys 0.028s
W20260709 13:16:04.712774 11517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.009s	sys 0.028s
W20260709 13:16:04.713814 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:04.711707 13956 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.005s	sys 0.027s
I20260709 13:16:04.656605 13877 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:04.721275 11397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.003s	sys 0.034s
W20260709 13:16:04.717147 11366 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.005s	sys 0.019s
W20260709 13:16:04.724045 11366 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.005s	sys 0.020s
W20260709 13:16:04.724558 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 1.53469e+06us
W20260709 13:16:04.717718 14027 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.003s
W20260709 13:16:04.719940 11441 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.007s	sys 0.021s
W20260709 13:16:04.728498 11441 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.007s	sys 0.021s
W20260709 13:16:04.728813 11366 connection.cc:256] RPC call timeout handler was delayed by -1.5217434260000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 13:16:04.730222 11443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.007s	sys 0.028s
W20260709 13:16:04.730607 11443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.007s	sys 0.029s
W20260709 13:16:04.725814 11397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.004s	sys 0.034s
I20260709 13:16:04.642069 14090 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:04.731387 11546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.608s	user 0.014s	sys 0.038s
W20260709 13:16:04.732040 11546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.608s	user 0.015s	sys 0.038s
W20260709 13:16:04.728811 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 691136us
W20260709 13:16:04.730903 11399 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.015s	sys 0.023s
W20260709 13:16:04.734942 11399 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.016s	sys 0.023s
W20260709 13:16:04.738670 11393 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.011s	sys 0.021s
W20260709 13:16:04.739964 11393 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.011s	sys 0.021s
W20260709 13:16:04.740839 11366 connection.cc:256] RPC call timeout handler was delayed by -1.0249434260001635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260709 13:16:04.743445 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:04.745543 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.722721 13877 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:04.772884 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1760) took 2246 ms. Trace:
I20260709 13:16:04.773913 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.774832 11472 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:04.775774 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:04.771409 13554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.007s	sys 0.030s
W20260709 13:16:04.779850 13554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.007s	sys 0.031s
I20260709 13:16:04.782313 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:04.783116 11474 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:04.784075 11476 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:04.748428 13877 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:04.749995 13989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.008s	sys 0.020s
W20260709 13:16:04.800498 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.024s	sys 0.047s
W20260709 13:16:04.795166 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 497784us
W20260709 13:16:04.836221 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.024s	sys 0.047s
W20260709 13:16:04.836637 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 996539us
I20260709 13:16:04.749025 13752 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.824296 11469 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:04.731860 14027 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.837399 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:04.839816 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:04.724773 13606 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:04.846447 11473 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:04.842453 11516 leader_election.cc:400] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:04.848255 11516 leader_election.cc:403] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:04.854229 14146 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:04.838730 13989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.020s
W20260709 13:16:04.850535 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.013s	sys 0.017s
I20260709 13:16:04.854727 14146 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:04.875705 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.014s	sys 0.017s
I20260709 13:16:04.878114 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.879420 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.883419 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:04.884162 11472 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:04.884223 11515 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.007s	sys 0.006s
W20260709 13:16:04.884783 11515 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.006s
W20260709 13:16:04.885175 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 710309us
I20260709 13:16:04.836494 14123 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:04.844928 13758 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:04.888456 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.889748 11468 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:16:04.898504 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:04.901082 11477 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:04.904687 11516 leader_election.cc:400] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:04.905198 11516 leader_election.cc:403] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:04.773696 11544 rpcz_store.cc:276] 0709 13:16:02.517510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.852706 (+335196us) service_pool.cc:224] Handling call
0709 13:16:02.937882 (+ 85176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.975473 (+ 37591us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:04.181792 (+1206319us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.763445 (+581653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:04.763528 (+    83us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:04.763596 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.763691 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.763854 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1560141,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2210,"raft.queue_time_us":21290,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":11887,"thread_start_us":1693033,"threads_started":2}
I20260709 13:16:04.882243 14146 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:04.932183 11297 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.014s	sys 0.001s
W20260709 13:16:04.932919 11297 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.014s	sys 0.001s
W20260709 13:16:04.937116 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50367.4us
I20260709 13:16:04.920121 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:04.901739 13802 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:16:04.953562 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:04.954650 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:04.955605 11402 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:04.912484 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 872999us
I20260709 13:16:04.859259 14124 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:16:04.962638 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:04.958702 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1237) took 1111 ms. Trace:
I20260709 13:16:04.954285 11398 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:04.986156 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:04.882308 13752 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.000144 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:05.001147 11400 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:16:05.005128 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 100594us
I20260709 13:16:04.909375 13984 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:16:05.004166 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.947882 13873 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:05.010589 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 282439us
W20260709 13:16:04.985232 14121 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 13:16:04.976287 14117 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:16:05.015116 11401 raft_consensus.cc:1217] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Deduplicated request from leader. Original: 5.3->[6.4-7.5]   Dedup: 7.5->[]
I20260709 13:16:05.031320 11401 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 13:16:04.982266 11394 rpcz_store.cc:276] 0709 13:16:03.833265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.912283 (+ 79018us) service_pool.cc:224] Handling call
0709 13:16:03.931456 (+ 19173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.931653 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:04.509144 (+577491us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.925526 (+416382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:04.925784 (+   258us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:04.943547 (+ 17763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.943805 (+   258us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.944456 (+   651us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1011644,"thread_start_us":936133,"threads_started":2}
I20260709 13:16:04.960515 13606 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:04.944379 13767 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 13:16:05.007581 14115 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.397s	user 0.000s	sys 0.002s
I20260709 13:16:04.934597 13758 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.038867 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1242) took 1187 ms. Trace:
W20260709 13:16:05.046844 11441 leader_election.cc:336] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.914s (SENT)
I20260709 13:16:05.056214 11472 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:16:05.048066 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:05.049099 13554 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:16:05.046909 14126 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:16:05.052269 14145 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:16:05.052711 14128 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:05.019372 14127 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:16:05.131625 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82725.8us
I20260709 13:16:05.072125 11393 rpcz_store.cc:276] 0709 13:16:03.834844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.919858 (+ 85014us) service_pool.cc:224] Handling call
0709 13:16:03.938089 (+ 18231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.956909 (+ 18820us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:04.742022 (+785113us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.021318 (+279296us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.021552 (+   234us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:05.021679 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.021788 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.021947 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9734,"prepare.queue_time_us":1071870,"raft.queue_time_us":80040,"spinlock_wait_cycles":22590720,"thread_start_us":987734,"threads_started":3}
W20260709 13:16:05.135286 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:05.124317 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 66553.6us
W20260709 13:16:05.139595 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 65687.8us
I20260709 13:16:05.141191 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1238) took 1307 ms. Trace:
I20260709 13:16:04.943692 14123 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.141707 11399 rpcz_store.cc:276] 0709 13:16:03.833489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.916691 (+ 83202us) service_pool.cc:224] Handling call
0709 13:16:03.917095 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.917656 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:04.737379 (+819723us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.140773 (+403394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.140837 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:05.140899 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.140981 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.141124 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22923,"prepare.queue_time_us":926626,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":52867,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":107,"spinlock_wait_cycles":8960,"thread_start_us":1162643,"threads_started":3}
I20260709 13:16:05.074611 13950 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:16:05.072294 14064 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:05.161140 13877 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:05.172730 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 234336us
W20260709 13:16:05.163780 14157 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:16:05.166152 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1735) took 2841 ms. Trace:
I20260709 13:16:05.178900 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1235) took 1346 ms. Trace:
I20260709 13:16:05.179589 11549 rpcz_store.cc:276] 0709 13:16:02.324695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.392449 (+ 67754us) service_pool.cc:224] Handling call
0709 13:16:02.393138 (+   689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:02.393750 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.393802 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:02.403339 (+  9537us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.167420 (+1764081us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:04.167464 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.168658 (+  1194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.165870 (+997212us) raft_consensus.cc:1685] finished
0709 13:16:05.165896 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.166078 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1658952,"threads_started":1,"wal-append.queue_time_us":2084705}
I20260709 13:16:05.120302 14156 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:05.129808 14128 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:05.208909 11402 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:05.213240 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1777) took 2490 ms. Trace:
I20260709 13:16:05.179636 11397 rpcz_store.cc:276] 0709 13:16:03.832609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.832712 (+   103us) service_pool.cc:224] Handling call
0709 13:16:03.861131 (+ 28419us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:03.861394 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.861992 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.862037 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.862701 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.731469 (+868768us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:04.731527 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.731620 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.935845 (+204225us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b18003c88c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:05.178672 (+242827us) raft_consensus.cc:1685] finished
0709 13:16:05.178697 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.178831 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28071,"raft.queue_time_us":1259,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":9812,"spinlock_wait_cycles":28075520,"thread_start_us":850721,"threads_started":1,"wal-append.queue_time_us":1010131}
I20260709 13:16:05.198813 11543 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:05.214512 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:05.215404 11543 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:05.179468 13802 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:05.224807 11472 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:05.230862 11543 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.235853 11543 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:05.213685 11475 rpcz_store.cc:276] 0709 13:16:02.722624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.910754 (+188130us) service_pool.cc:224] Handling call
0709 13:16:02.911198 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:02.911455 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.911484 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:02.977444 (+ 65960us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.203145 (+1225701us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:04.203186 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.203263 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.212952 (+1009689us) raft_consensus.cc:1685] finished
0709 13:16:05.212979 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.213149 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1174021,"threads_started":1,"wal-append.queue_time_us":1502149}
I20260709 13:16:05.238569 11469 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:16:05.233753 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1685) took 2467 ms. Trace:
I20260709 13:16:05.213481 11365 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:05.246624 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 71953.6us
I20260709 13:16:05.211323 14064 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:05.188542 14156 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:05.240754 11396 rpcz_store.cc:276] 0709 13:16:02.766575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.189481 (+422906us) service_pool.cc:224] Handling call
0709 13:16:03.256312 (+ 66831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.284044 (+ 27732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.284086 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.461236 (+177150us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.486941 (+1025705us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:04.487024 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.487120 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.233479 (+746359us) raft_consensus.cc:1685] finished
0709 13:16:05.233502 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.233684 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":954948,"threads_started":1,"wal-append.queue_time_us":1240661}
I20260709 13:16:05.208772 14128 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:05.248930 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 131486us
I20260709 13:16:05.255060 14123 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:05.255786 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1789) took 2060 ms. Trace:
I20260709 13:16:05.255975 14123 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:05.256188 11471 rpcz_store.cc:276] 0709 13:16:03.195203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.838919 (+643716us) service_pool.cc:224] Handling call
0709 13:16:03.871412 (+ 32493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.871638 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.871660 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.872112 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.649553 (+777441us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:04.649606 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.649711 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.255572 (+605861us) raft_consensus.cc:1685] finished
0709 13:16:05.255591 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.255728 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50816,"thread_start_us":777006,"threads_started":1,"wal-append.queue_time_us":1341778}
I20260709 13:16:05.258438 14090 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:05.260314 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1766) took 2690 ms. Trace:
I20260709 13:16:05.265695 11477 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:05.183131 13818 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:16:05.237727 14027 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:05.276937 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.260726 11546 rpcz_store.cc:276] 0709 13:16:02.586248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.896353 (+310105us) service_pool.cc:224] Handling call
0709 13:16:03.027120 (+130767us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.027728 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.027781 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.113915 (+ 86134us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.732733 (+1618818us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:04.732773 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.732864 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.260079 (+527215us) raft_consensus.cc:1685] finished
0709 13:16:05.260102 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.260250 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8850,"raft.queue_time_us":1859,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":43203,"thread_start_us":1609021,"threads_started":1,"wal-append.queue_time_us":1773922}
W20260709 13:16:05.323944 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 110529us
I20260709 13:16:05.325189 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:05.346510 14132 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:05.347136 14132 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:05.348261 14132 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.349584 14132 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:05.379592 13902 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:05.380529 13984 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.381084 13984 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:05.383630 14155 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:16:05.250712 14156 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:05.386504 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.387187 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:05.395490 14143 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:16:05.407440 12119 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:05.407898 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.408727 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:05.276011 14123 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:05.398018 14133 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.474903 11543 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:05.412479 11469 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:05.285964 14064 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:05.494823 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.410497 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 302) took 1085 ms. Trace:
I20260709 13:16:05.409039 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:05.507447 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69761.1us
I20260709 13:16:05.510064 12112 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:16:05.440474 13763 consensus_meta.cc:287] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.157s	user 0.009s	sys 0.005s
I20260709 13:16:05.488447 13554 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:05.504284 11545 rpcz_store.cc:276] 0709 13:16:04.325231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.326145 (+   914us) service_pool.cc:224] Handling call
0709 13:16:04.326483 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.706892 (+380409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.706936 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.728994 (+ 22058us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.801326 (+ 72332us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:04.801682 (+   356us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.801924 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.802028 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.387131 (+585103us) raft_consensus.cc:1685] finished
0709 13:16:05.387153 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.410413 (+ 23260us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":677,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":265248,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":4618,"thread_start_us":450634,"threads_started":2,"wal-append.queue_time_us":426533}
I20260709 13:16:05.407493 14147 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.441533 13606 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.443711 14116 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:05.554720 14116 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.557053 14116 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.444751 14160 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:05.557744 14160 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.559868 14160 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:05.528733 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.560642 12117 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:05.561684 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.496346 14133 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.564714 14133 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.532253 14064 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.533001 13877 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:05.441327 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.533641 14147 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.510462 14027 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:05.442499 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.535851 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.538166 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.539275 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.540179 14178 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.541615 13818 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:05.525120 13763 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.543300 14165 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:05.543740 13554 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:05.546759 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 296) took 1226 ms. Trace:
I20260709 13:16:05.444605 14168 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.554750 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:05.566275 11473 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.001s
I20260709 13:16:05.438369 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:05.441829 13752 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:05.514940 14123 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:05.676441 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 400698us
W20260709 13:16:05.599383 11474 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.002s
I20260709 13:16:05.716362 11474 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:16:05.636479 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 351183us
W20260709 13:16:05.715301 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.717813 11473 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:05.597084 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.715410 11551 rpcz_store.cc:276] 0709 13:16:04.320252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.322372 (+  2120us) service_pool.cc:224] Handling call
0709 13:16:04.324017 (+  1645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.324398 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.324434 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.325073 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.798521 (+473448us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:04.799164 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.799990 (+   826us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.546533 (+746543us) raft_consensus.cc:1685] finished
0709 13:16:05.546556 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.546701 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200863,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4863,"thread_start_us":471417,"threads_started":1,"wal-append.queue_time_us":559262}
W20260709 13:16:05.567835 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 242703us
W20260709 13:16:05.638183 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 590162us
W20260709 13:16:05.696550 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.027s	sys 0.023s
I20260709 13:16:05.676941 14138 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.755416 11515 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:05.566179 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.635578 13950 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
W20260709 13:16:05.760996 11440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.030s	sys 0.020s
W20260709 13:16:05.761363 11440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.030s	sys 0.020s
W20260709 13:16:05.751042 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.029s	sys 0.023s
I20260709 13:16:05.753911 12115 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:05.593917 13606 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.767036 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:05.730719 14168 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.656618 13984 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.760179 13921 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:16:05.767925 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.825421 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:05.833256 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.787092 12085 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:05.837404 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 70463us
I20260709 13:16:05.836088 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:05.832036 14090 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:05.765800 14174 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:05.761557 14165 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:05.848148 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 124694us
I20260709 13:16:05.732385 14178 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.822778 14099 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:05.850286 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 109275us
I20260709 13:16:05.851490 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.852362 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:05.846390 14128 consensus_meta.cc:287] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:16:05.767972 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.882113 13763 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.838060 13818 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:05.882174 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:05.882767 13818 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:05.883641 13818 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.883963 13921 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:05.848409 14174 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.884940 14174 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:05.888343 14168 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.814755 14156 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.845927 13767 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:05.849382 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:05.872026 12116 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:05.873565 13606 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.764117 14138 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.875552 14084 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:05.875159 13554 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.846655 13877 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:05.876322 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89331us
I20260709 13:16:05.877038 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.879400 14147 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:05.879985 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:05.880808 14128 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:05.889758 11469 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:05.846870 13950 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:05.847922 14090 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:05.835865 14177 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
W20260709 13:16:05.961544 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 69865.5us
I20260709 13:16:06.011206 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.030701 11468 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:05.890918 11475 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
W20260709 13:16:05.970752 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 121392us
I20260709 13:16:06.010924 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.030625 12117 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7. Re-sending same reply.
W20260709 13:16:05.963245 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 72735.4us
W20260709 13:16:05.944437 14146 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.995s	user 0.011s	sys 0.000s
I20260709 13:16:06.037894 14123 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.039685 14090 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:05.966970 13989 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.040377 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:06.040544 14173 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:06.019708 14190 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:06.054340 14190 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:06.054750 14190 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.055131 14178 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:06.055428 14132 consensus_meta.cc:287] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.705s	user 0.013s	sys 0.001s
I20260709 13:16:06.056998 14181 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.057513 13989 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:06.059535 14140 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.615s	user 0.000s	sys 0.004s
I20260709 13:16:06.059733 13989 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:05.882368 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.060676 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:06.061203 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.043242 14138 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.061836 12115 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:06.062022 14138 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:05.920882 14144 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:05.868254 14027 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.063392 14144 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:06.063846 13802 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:06.064291 13802 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:06.064133 14027 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.064412 14144 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.065474 14090 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:06.065279 13802 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.066002 14090 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.066548 13802 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.015096 14172 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:06.067233 14172 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.069404 14172 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:06.070626 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.048166 12116 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:05.944936 14100 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.051353 12123 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:06.044683 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:05.965998 14170 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.056849 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.052421 14189 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:06.062778 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.072257 12117 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:05.944793 14179 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 13:16:05.920258 14064 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.159s
I20260709 13:16:05.890928 13767 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.056880 14128 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.084213 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:06.019316 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:06.086802 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.090281 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:06.092497 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.093523 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:06.089056 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 70756.3us
I20260709 13:16:06.083081 14172 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:06.050017 14146 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.100466 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.083128 12114 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:06.101629 11441 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:06.066926 14090 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.100545 14132 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:06.108279 13767 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.085739 14189 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.082835 14160 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:16:06.100139 14156 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.035483 14165 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.123798 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.057559 14181 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.125970 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.080694 14170 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.082458 14173 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.115876 14132 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:06.115635 14172 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:06.129223 14183 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:06.144892 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:06.143198 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 81701.1us
I20260709 13:16:06.145712 14190 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.133623 14064 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:06.149451 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:06.142928 14132 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:06.158721 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:06.154908 13767 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.164794 14190 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.142277 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.148698 14183 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.171722 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:06.173647 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:06.184643 11474 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:06.153749 14193 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:16:06.189296 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:06.190389 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:06.178167 13877 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:06.191048 11468 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:06.194732 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:06.195762 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:06.196746 11471 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:06.191377 13877 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.168474 13767 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:06.220664 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:06.221120 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.177898 13879 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:06.199939 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:06.201727 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:06.203058 14187 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:06.207768 14146 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:06.208453 14172 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.212083 14170 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.212940 13877 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:06.223557 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.186697 14183 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.231366 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.233037 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.233668 11546 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:06.191615 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.231456 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.237298 11544 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.238144 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.236203 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.198802 14123 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.155731 14174 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.240105 11547 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:16:06.243275 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.225368 14178 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.219856 14003 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:06.270689 11551 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:06.271356 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.275717 11546 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:06.224794 13767 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.239230 14191 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.236042 13879 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.236379 14187 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:06.303411 14142 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.168s	user 0.008s	sys 0.001s
W20260709 13:16:06.304263 14157 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.067s	user 0.005s	sys 0.001s
I20260709 13:16:06.271423 14123 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:06.293883 14130 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:16:06.313171 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.318681 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.318328 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.314641 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:06.277784 14003 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:06.297964 14191 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.311316 14181 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.319482 14123 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:06.373674 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72520.7us
I20260709 13:16:06.386637 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:06.282236 14178 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.376883 11477 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:16:06.387966 11549 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:06.377166 14174 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.383329 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.392342 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 303) took 2066 ms. Trace:
I20260709 13:16:06.369474 14191 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.405396 14178 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:06.406719 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.408082 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.408618 12116 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.369784 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.371420 14160 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.403383 13990 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:16:06.403493 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 88966.8us
I20260709 13:16:06.404858 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:06.394111 11548 rpcz_store.cc:276] 0709 13:16:04.325562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.326901 (+  1339us) service_pool.cc:224] Handling call
0709 13:16:04.327242 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.711261 (+384019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.711307 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.725420 (+ 14113us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.803141 (+ 77721us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:04.803489 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.803693 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.803783 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.303967 (+1500184us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b2400208ce8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686b09d 00007f8da949d10e 00007f8da94b488c 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221
0709 13:16:06.392067 (+ 88100us) raft_consensus.cc:1685] finished
0709 13:16:06.392150 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.392283 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1853,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":264276,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":58400,"spinlock_wait_cycles":53781376,"thread_start_us":420613,"threads_started":2,"wal-append.queue_time_us":166563}
I20260709 13:16:06.420185 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.397205 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.369827 14177 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:06.422288 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:06.424671 11516 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:06.427044 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:06.431941 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.433749 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.446547 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.437508 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.449524 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.419889 14160 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:06.451187 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64627.3us
I20260709 13:16:06.419288 13990 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.451912 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:06.452859 11396 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:06.454295 11548 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.455550 11402 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:06.457114 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:06.431808 14084 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:06.458564 11400 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:06.461287 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.460273 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:06.443923 14027 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:06.417881 12114 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:06.436697 14174 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:06.468700 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:06.465991 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 63150.3us
I20260709 13:16:06.431003 13879 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.429477 14178 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:06.471712 11399 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:06.472576 11477 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:06.477655 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.443769 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.459512 14138 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:06.459893 14160 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.431187 14156 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:16:06.461128 14084 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.465004 14190 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:06.493297 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.493535 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.494377 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.471490 14174 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:06.468046 13950 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:06.484002 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.473196 14027 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:06.478498 14180 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.478466 14192 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:06.477353 14178 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.498289 11400 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:06.520869 14191 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.522221 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.494604 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.522873 13758 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:06.523233 11399 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:06.523341 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:06.501382 11398 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:06.481473 14193 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:06.511807 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78970.8us
I20260709 13:16:06.495708 14189 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.505973 12116 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:06.488778 14181 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:06.451618 14187 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.496202 14084 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.521309 14178 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.524114 11402 raft_consensus.cc:2368] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:06.487569 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:06.524183 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.522922 11400 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:06.453896 14003 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.503355 14174 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:06.503501 14190 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.586742 11397 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.542454 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.510751 14027 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.541886 14181 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:06.571339 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 111135us
I20260709 13:16:06.586685 11393 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:06.596226 11399 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:06.543262 12117 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:06.611433 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 83436.7us
I20260709 13:16:06.611563 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.512488 13950 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:06.623796 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.588321 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.601426 14156 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:06.624994 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:06.527798 14180 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.627846 11544 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:06.520857 14132 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:06.662045 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.567744 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:06.542421 14191 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.667290 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.667701 14132 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.667987 12113 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8. Re-sending same reply.
W20260709 13:16:06.668326 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.669023 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.671536 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.672643 11546 raft_consensus.cc:2410] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:16:06.639644 14151 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.197s	user 0.003s	sys 0.000s
I20260709 13:16:06.653612 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.673749 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.674108 12115 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:06.674578 11546 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:06.633100 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:06.624926 14181 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:06.635929 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:06.637046 14207 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:16:06.637838 14209 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:06.638412 14190 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.639729 13950 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.704967 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.644457 11397 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:06.645625 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1778) took 4046 ms. Trace:
I20260709 13:16:06.652918 14114 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:06.568075 14144 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:16:06.659282 11400 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:06.528477 14183 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.675158 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.612488 14003 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.719997 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:06.720855 11546 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:16:06.625566 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:06.676548 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:06.676955 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 196858us
I20260709 13:16:06.677078 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:06.722431 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45453us
W20260709 13:16:06.722916 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 97421.7us
I20260709 13:16:06.669615 12113 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8. Re-sending same reply.
I20260709 13:16:06.650477 13767 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.682201 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:06.642673 13990 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.728164 11544 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.729977 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:06.711475 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78254.3us
I20260709 13:16:06.708637 11543 rpcz_store.cc:276] 0709 13:16:02.606116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.126245 (+520129us) service_pool.cc:224] Handling call
0709 13:16:03.126601 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.484466 (+2357865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.645359 (+1160893us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54001a6dd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:06.645561 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4729322240}
I20260709 13:16:06.714675 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.718731 11548 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:06.713845 14156 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:06.724658 11516 leader_election.cc:400] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:06.733084 11516 leader_election.cc:403] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:16:06.733067 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:06.728204 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:06.711316 14114 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:06.733939 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:06.735282 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:06.736366 11542 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:06.739789 11549 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:06.727897 14192 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:06.729576 14146 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.751291 14146 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:06.751431 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:06.751765 14146 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.733433 14156 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:06.752264 14156 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:06.755393 14172 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:06.755317 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.755927 14172 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:06.756269 12113 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:06.757249 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.756327 14172 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:06.747299 14213 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:06.761142 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 337) took 2642 ms. Trace:
I20260709 13:16:06.761699 13802 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:06.762359 13802 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.761533 11395 rpcz_store.cc:276] 0709 13:16:04.118855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.290440 (+171585us) service_pool.cc:224] Handling call
0709 13:16:04.290749 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.299177 (+  8428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.299219 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.319758 (+ 20539us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.803223 (+483465us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:05.133960 (+330737us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.139421 (+  5461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.139499 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.760035 (+1620536us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b2400137078. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:16:06.760863 (+   828us) raft_consensus.cc:1685] finished
0709 13:16:06.760952 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.761079 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7861,"prepare.queue_time_us":363878,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":14284,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":21939,"spinlock_wait_cycles":77742336,"thread_start_us":786275,"threads_started":2,"wal-append.queue_time_us":655718}
I20260709 13:16:06.766119 14173 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:06.762910 13802 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.745174 11545 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:06.736593 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.771714 14183 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:06.772614 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.775414 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.776602 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.776649 14027 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.777456 12122 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:06.778028 11545 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:06.778223 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.780431 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 389) took 1395 ms. Trace:
I20260709 13:16:06.780514 14180 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:06.780908 12123 rpcz_store.cc:276] 0709 13:16:05.384778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.384954 (+   176us) service_pool.cc:224] Handling call
0709 13:16:05.385364 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.073435 (+688071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.780177 (+706742us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54001d0bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:06.780329 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525910912}
I20260709 13:16:06.758435 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:06.782390 12113 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.782841 14162 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:06.783252 14162 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:06.784071 14162 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:06.785745 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.715531 14214 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:16:06.747864 14114 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:06.801084 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:06.803042 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1776) took 4080 ms. Trace:
I20260709 13:16:06.803510 11470 rpcz_store.cc:276] 0709 13:16:02.722400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.907932 (+185532us) service_pool.cc:224] Handling call
0709 13:16:02.908341 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:02.908610 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.908639 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:02.909184 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.174450 (+1265266us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:04.174494 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.174571 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.781915 (+2607344us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b2400296bb8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:16:06.785056 (+  3141us) raft_consensus.cc:1685] finished
0709 13:16:06.785078 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.802931 (+ 17853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43677056,"thread_start_us":1264777,"threads_started":1,"wal-append.queue_time_us":1628145}
I20260709 13:16:06.817442 14114 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.821390 14123 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:06.822443 14184 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.950s	user 0.004s	sys 0.000s
I20260709 13:16:06.823822 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 398) took 1415 ms. Trace:
I20260709 13:16:06.823872 14123 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:06.824683 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.826443 11550 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:06.824229 12118 rpcz_store.cc:276] 0709 13:16:05.408496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.409285 (+   789us) service_pool.cc:224] Handling call
0709 13:16:05.495040 (+ 85755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.536659 (+ 41619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.536690 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.537174 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.714295 (+177121us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:05.714353 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.714447 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.821993 (+1107546us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b24003e3738. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686b09d 00007f8da949d10e 00007f8da94b488c 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221
0709 13:16:06.823409 (+  1416us) raft_consensus.cc:1685] finished
0709 13:16:06.823436 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.823639 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3393,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":27215104,"thread_start_us":176673,"threads_started":1,"wal-append.queue_time_us":294538}
I20260709 13:16:06.828517 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.815236 14162 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.844700 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:06.842895 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 430) took 1306 ms. Trace:
I20260709 13:16:06.745865 14216 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:06.843654 14146 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.738970 14191 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:16:06.815570 14100 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 13:16:07.008998 12116 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:07.134755 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 383) took 1257 ms. Trace:
I20260709 13:16:06.902848 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:07.121269 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 142040us
I20260709 13:16:06.920056 11544 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
W20260709 13:16:07.138244 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.879612 12118 rpcz_store.cc:276] 0709 13:16:05.536270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.828261 (+1291991us) service_pool.cc:224] Handling call
0709 13:16:06.842759 (+ 14498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.132495 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:07.138103 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 337127us
W20260709 13:16:07.132220 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 399212us
I20260709 13:16:07.147495 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 449) took 1589 ms. Trace:
I20260709 13:16:07.148008 12122 rpcz_store.cc:276] 0709 13:16:05.558146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.935394 (+1377248us) service_pool.cc:224] Handling call
0709 13:16:07.147437 (+212043us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:07.139699 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.134816 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:07.004895 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 271023us
I20260709 13:16:07.150345 12112 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:07.150923 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 507) took 1092 ms. Trace:
I20260709 13:16:06.828917 14189 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:06.815696 14128 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:07.151266 12112 rpcz_store.cc:276] 0709 13:16:06.058205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.829266 (+771061us) service_pool.cc:224] Handling call
0709 13:16:07.150869 (+321603us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:07.152762 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.135361 11473 rpcz_store.cc:276] 0709 13:16:05.877799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.877984 (+   185us) service_pool.cc:224] Handling call
0709 13:16:05.878313 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.032239 (+153926us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:06.120225 (+ 87986us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.172985 (+ 52760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.173065 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:06.173137 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.134482 (+961345us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:07.134536 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.134684 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":99232,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":243,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9580,"spinlock_wait_cycles":346090880,"thread_start_us":106442,"threads_started":2,"wal-append.queue_time_us":80628}
I20260709 13:16:06.986420 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:07.078107 14220 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 13:16:06.924484 14219 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.341s
I20260709 13:16:07.156965 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:07.157008 14003 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:07.160415 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.161645 11548 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:06.778838 14027 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.163460 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:07.149003 13758 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:07.170123 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:07.166443 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.166370 11544 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:07.138842 13767 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:07.152709 14132 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.082193 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:07.173720 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.172359 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:07.176357 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:07.177482 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:07.076361 14181 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.174672 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.025588 14174 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:06.985571 14084 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.040926 14156 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.178491 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:07.180773 14180 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:07.173595 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 408) took 1349 ms. Trace:
I20260709 13:16:07.177866 14132 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.186935 12112 rpcz_store.cc:276] 0709 13:16:05.824598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.156479 (+1331881us) service_pool.cc:224] Handling call
0709 13:16:07.173539 (+ 17060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.188046 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.188887 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 450) took 1630 ms. Trace:
I20260709 13:16:07.189266 12112 rpcz_store.cc:276] 0709 13:16:05.558410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.187766 (+1629356us) service_pool.cc:224] Handling call
0709 13:16:07.188835 (+  1069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.190179 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.190907 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 520) took 1129 ms. Trace:
I20260709 13:16:07.191318 12112 rpcz_store.cc:276] 0709 13:16:06.061916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.189946 (+1128030us) service_pool.cc:224] Handling call
0709 13:16:07.190857 (+   911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.192227 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.193113 12112 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:07.193586 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 513) took 1133 ms. Trace:
I20260709 13:16:07.193933 12112 rpcz_store.cc:276] 0709 13:16:06.059981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.192000 (+1132019us) service_pool.cc:224] Handling call
0709 13:16:07.193536 (+  1536us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.176268 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:07.177611 14123 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:07.179620 14214 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:07.179888 14183 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:07.180351 14219 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:07.182483 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 526) took 1082 ms. Trace:
I20260709 13:16:07.048560 13802 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.164470 14216 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:07.219762 12123 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:06.985177 14160 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.181222 12118 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:07.171481 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.172349 13950 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:06.985849 14172 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.174738 14064 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:07.186249 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 51520.3us
I20260709 13:16:07.234457 14172 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.194836 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.196180 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:06.994184 14178 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.198071 14183 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:07.198411 14214 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:07.197213 14114 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:07.183288 14226 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:07.198463 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:07.217234 11545 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:07.222169 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 428) took 1702 ms. Trace:
W20260709 13:16:07.198848 14233 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:16:07.231879 11473 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:07.232956 13767 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:07.242558 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 402) took 1832 ms. Trace:
I20260709 13:16:07.245671 11398 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:07.252934 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54541.6us
I20260709 13:16:07.253218 12118 rpcz_store.cc:276] 0709 13:16:05.519810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.164604 (+1644794us) service_pool.cc:224] Handling call
0709 13:16:07.222098 (+ 57494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.254638 11544 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:07.220781 14224 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:07.256762 12112 rpcz_store.cc:276] 0709 13:16:05.409855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.194629 (+1784774us) service_pool.cc:224] Handling call
0709 13:16:07.242476 (+ 47847us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.259728 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.257550 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:07.257670 11545 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:16:07.260667 12112 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:07.217849 12113 rpcz_store.cc:276] 0709 13:16:06.102013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.758176 (+656163us) service_pool.cc:224] Handling call
0709 13:16:07.181777 (+423601us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.230644 13950 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.225096 14027 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:07.235530 14064 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:07.261451 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 411) took 1168 ms. Trace:
I20260709 13:16:07.239722 14183 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:07.263548 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 94181.6us
I20260709 13:16:07.266316 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.266522 12112 rpcz_store.cc:276] 0709 13:16:06.093337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.259491 (+1166154us) service_pool.cc:224] Handling call
0709 13:16:07.261400 (+  1909us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:07.266831 11514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.184s (SENT) [suppressed 42 similar messages]
I20260709 13:16:07.267755 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.244400 14219 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:07.264006 11544 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:07.251019 14214 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:07.247433 14114 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.254417 14172 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:07.259186 14224 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.271802 12118 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:07.277774 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 533) took 1061 ms. Trace:
I20260709 13:16:07.278303 12112 rpcz_store.cc:276] 0709 13:16:06.210735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.267449 (+1056714us) service_pool.cc:224] Handling call
0709 13:16:07.271843 (+  4394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.262638 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 531) took 1051 ms. Trace:
I20260709 13:16:07.265633 13950 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:07.280514 14216 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.279740 14224 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.291853 13802 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:07.292901 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:07.292763 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:07.324911 11514 connection.cc:256] RPC call timeout handler was delayed by -1.1717431699998997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 13:16:07.325500 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:07.325784 11514 connection.cc:256] RPC call timeout handler was delayed by -1.1060431699997935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 13:16:07.326082 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0439431699999204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 13:16:07.326350 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0034431699998549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 13:16:07.295733 11544 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:07.296478 14219 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:07.296969 14174 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:07.297526 11545 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:07.297924 14156 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.298784 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.336310 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.299902 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.337924 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:07.338518 14229 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:16:07.338729 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.339345 12114 raft_consensus.cc:2376] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8. Re-sending same reply.
I20260709 13:16:07.339612 11550 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:07.340255 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.340641 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.341383 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.341771 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:07.342418 12114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:56142
I20260709 13:16:07.305912 14052 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:07.343248 14052 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.321718 14213 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:16:07.344789 14128 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:07.345284 14128 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:07.345470 14052 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.346035 14084 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.308732 12116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 452) took 1184 ms. Trace:
I20260709 13:16:07.346158 14128 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.346771 12116 rpcz_store.cc:276] 0709 13:16:06.124844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.607424 (+482580us) service_pool.cc:224] Handling call
0709 13:16:07.308467 (+701043us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400720454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:07.308621 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588583680}
I20260709 13:16:07.345306 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1797) took 1119 ms. Trace:
I20260709 13:16:07.347453 14128 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.271253 14064 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.347715 11543 rpcz_store.cc:276] 0709 13:16:06.225398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.731457 (+506059us) service_pool.cc:224] Handling call
0709 13:16:06.740079 (+  8622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.740384 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.740427 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.815974 (+ 75547us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.256730 (+440756us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:07.256773 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.256843 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.345106 (+ 88263us) raft_consensus.cc:1685] finished
0709 13:16:07.345127 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.345249 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":95878,"threads_started":1,"wal-append.queue_time_us":127924}
I20260709 13:16:07.312335 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.315877 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 534) took 1104 ms. Trace:
W20260709 13:16:07.317715 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 60221.5us
I20260709 13:16:07.350033 12118 rpcz_store.cc:276] 0709 13:16:06.211066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.261115 (+1050049us) service_pool.cc:224] Handling call
0709 13:16:07.315762 (+ 54647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.350464 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:07.351397 12118 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:16:07.351616 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.351971 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 404) took 1600 ms. Trace:
W20260709 13:16:07.352104 12117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 433) took 1815 ms (client timeout 2133 ms). Trace:
I20260709 13:16:07.352452 11393 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.352378 12118 rpcz_store.cc:276] 0709 13:16:05.751470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.350726 (+1599256us) service_pool.cc:224] Handling call
0709 13:16:07.351047 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.351730 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.351901 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.354228 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 510) took 1295 ms. Trace:
I20260709 13:16:07.338744 11396 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.260092 14226 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.356163 14181 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.357396 14226 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:07.358618 13984 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:07.344898 13877 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:07.359105 13984 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:07.360129 13984 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.360963 14177 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:07.361359 14177 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:07.361464 13984 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.361764 14132 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:07.275894 14114 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:07.366760 14231 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:16:07.368522 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1804) took 1140 ms. Trace:
I20260709 13:16:07.319613 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:07.333848 11542 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:07.271438 14218 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:07.293378 14178 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:07.363837 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.362287 14177 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.364328 14146 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:07.363960 14219 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.399678 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.324018 14173 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:16:07.366065 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 401) took 1956 ms. Trace:
W20260709 13:16:07.425108 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 74714.4us
I20260709 13:16:07.426151 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:07.368854 11546 rpcz_store.cc:276] 0709 13:16:06.228589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.744517 (+515928us) service_pool.cc:224] Handling call
0709 13:16:06.815824 (+ 71307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.816056 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.816088 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.828103 (+ 12015us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.159265 (+331162us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:07.159295 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.159341 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.368315 (+208974us) raft_consensus.cc:1685] finished
0709 13:16:07.368333 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.368462 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":330819,"threads_started":1,"wal-append.queue_time_us":370369}
W20260709 13:16:07.370252 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50715.1us
I20260709 13:16:07.281175 12115 rpcz_store.cc:276] 0709 13:16:06.210160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.803223 (+593063us) service_pool.cc:224] Handling call
0709 13:16:07.261499 (+458276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:07.352476 12117 rpcz_store.cc:269] 0709 13:16:05.537129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.092324 (+555195us) service_pool.cc:224] Handling call
0709 13:16:07.351961 (+1259637us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54005ff0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:07.352053 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290690304}
I20260709 13:16:07.354552 12118 rpcz_store.cc:276] 0709 13:16:06.059086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.353266 (+1294180us) service_pool.cc:224] Handling call
0709 13:16:07.353577 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.353876 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.353901 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.353940 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:07.353983 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.354040 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.354142 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1488,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73}
I20260709 13:16:07.354743 11393 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:07.377251 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.381085 11474 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:07.397735 11547 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:07.398265 11475 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:07.372627 14218 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:07.420385 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:07.411715 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1803) took 1185 ms. Trace:
I20260709 13:16:07.444052 11549 rpcz_store.cc:276] 0709 13:16:06.226271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.389229 (+162958us) service_pool.cc:224] Handling call
0709 13:16:07.402597 (+1013368us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b540033c2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:07.411659 (+  9062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726418688}
I20260709 13:16:07.412109 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.412400 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.395210 14170 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.416508 14204 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:07.417426 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.417847 11469 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:07.407905 14183 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:07.419317 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.423270 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.440229 11400 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.440644 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:07.451538 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:07.456621 11474 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.445103 11396 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:07.429180 11367 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:07.318245 14027 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:07.447973 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:07.452399 11471 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:07.453935 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.462493 11549 raft_consensus.cc:2368] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:16:07.455458 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 516) took 1385 ms. Trace:
I20260709 13:16:07.463439 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.424342 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.463215 12117 rpcz_store.cc:276] 0709 13:16:06.060829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.436200 (+1375371us) service_pool.cc:224] Handling call
0709 13:16:07.436581 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.445699 (+  9118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.445741 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.445791 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:07.445822 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.445894 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.446003 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.464720 11401 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.465536 11475 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:07.452476 14052 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.467476 12119 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:07.469437 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 514) took 1409 ms. Trace:
I20260709 13:16:07.469777 12119 rpcz_store.cc:276] 0709 13:16:06.060246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.424712 (+1364466us) service_pool.cc:224] Handling call
0709 13:16:07.454101 (+ 29389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.469168 (+ 15067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.469362 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.469110 14114 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.441284 13877 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:07.472227 14114 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:07.473297 12116 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:07.445650 14156 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:16:07.473560 11396 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
W20260709 13:16:07.474519 14230 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:16:07.474756 11472 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:07.475098 12114 raft_consensus.cc:606] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:07.446760 14218 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:07.475485 12114 raft_consensus.cc:614] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:07.473898 14156 consensus_queue.cc:1243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:07.424355 12113 rpcz_store.cc:276] 0709 13:16:05.409520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.277672 (+1868152us) service_pool.cc:224] Handling call
0709 13:16:07.365999 (+ 88327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.524832 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:07.526572 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1251) took 1065 ms. Trace:
I20260709 13:16:07.527858 12112 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:07.515980 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.529294 11549 raft_consensus.cc:2393] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.492108 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.522886 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 413) took 1137 ms. Trace:
I20260709 13:16:07.515760 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.530344 12123 rpcz_store.cc:276] 0709 13:16:06.385138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.165265 (+780127us) service_pool.cc:224] Handling call
0709 13:16:07.522676 (+357411us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b54005cf654. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:07.522828 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70273792}
I20260709 13:16:07.527560 11398 rpcz_store.cc:276] 0709 13:16:06.462144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.531800 (+ 69656us) service_pool.cc:224] Handling call
0709 13:16:06.588173 (+ 56373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.257041 (+668868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.526271 (+269230us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54003d4fd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:07.526498 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511701376}
I20260709 13:16:07.456943 14177 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:07.532958 11474 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
W20260709 13:16:07.503602 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.533394 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.535768 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:07.536657 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 215898us
I20260709 13:16:07.456125 14064 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.491176 14183 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:07.502954 14162 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:07.542912 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.522245 14215 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:07.562940 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.553886 11474 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:07.540607 11469 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:07.564337 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.562659 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:07.569451 11469 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:16:07.569940 11551 raft_consensus.cc:2410] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:07.571137 11469 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:07.554949 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.557237 11476 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:07.575067 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.575433 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:07.575819 12118 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.563519 11441 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:07.578387 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.579135 11549 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.575192 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.581632 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.581944 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:07.582340 11402 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:07.583269 12117 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:07.589641 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 160551us
I20260709 13:16:07.559631 14162 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.592518 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:07.593194 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.594352 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.592402 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:07.595369 11549 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:16:07.598289 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 150406us
I20260709 13:16:07.564261 13984 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:16:07.598961 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.731890 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:07.750792 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.966447 12116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 434) took 2429 ms. Trace:
I20260709 13:16:07.732684 11401 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:08.004479 12116 rpcz_store.cc:276] 0709 13:16:05.537442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.347695 (+1810253us) service_pool.cc:224] Handling call
0709 13:16:07.347987 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.473750 (+125763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.966206 (+492456us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540021dc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:07.966390 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287669120}
I20260709 13:16:07.717160 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.005638 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.885397 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.887202 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:07.599298 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:08.005076 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.731295 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.908428 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 442) took 2358 ms. Trace:
W20260709 13:16:07.618714 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 383677us
I20260709 13:16:08.009946 11476 raft_consensus.cc:2368] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
W20260709 13:16:08.015618 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.931195 11400 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:07.964455 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.016695 11474 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.018265 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.684459 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.020206 11474 raft_consensus.cc:2368] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:07.686882 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.021471 11545 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:07.936774 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:07.901306 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 337820us
W20260709 13:16:08.008764 13950 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.000s
I20260709 13:16:07.597795 14173 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.013837 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:07.691046 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:08.000691 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 377367us
I20260709 13:16:07.716943 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:07.738660 14216 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.265s
I20260709 13:16:08.009223 11549 raft_consensus.cc:2368] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:16:08.003181 11398 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.905128 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.018946 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.045159 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.045784 11400 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:07.907532 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.717749 14160 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:07.598187 14162 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:07.852705 14237 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.034724 13902 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.054039 13902 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.056118 13902 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:08.057219 14254 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:16:08.023361 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:07.727820 14245 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:16:07.922631 14251 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:16:08.061159 14100 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.061615 14100 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:08.024514 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.063938 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.064513 11551 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.024837 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.067894 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.068940 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.069914 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:07.691351 14171 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.604s
I20260709 13:16:08.071552 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.026768 11441 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:07.594408 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:07.962463 14214 consensus_meta.cc:287] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.000s
I20260709 13:16:08.082159 14214 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.084307 14214 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:08.086184 11551 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:08.030283 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.095775 11474 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.096762 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.097900 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.098341 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:07.691365 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 372079us
I20260709 13:16:07.596963 14183 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.042299 11543 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:08.037979 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:07.595018 14204 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:07.599246 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:07.958680 14255 raft_consensus.cc:476] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Not starting pre-election -- already a leader
I20260709 13:16:07.928975 14027 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:07.896379 11548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.004s	sys 0.007s
W20260709 13:16:08.111820 11548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.005s	sys 0.007s
I20260709 13:16:08.049937 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.112599 11549 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:08.050602 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.112831 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.113547 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.114904 11400 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:07.862656 14064 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:07.597592 14181 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.448s
I20260709 13:16:07.597322 14190 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.449s
I20260709 13:16:08.075575 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:08.077076 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50422.2us
I20260709 13:16:08.077293 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.117669 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.118432 11545 raft_consensus.cc:2410] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:08.080190 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.119939 11395 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:08.121109 11395 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:16:08.081645 12113 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:08.087158 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.123240 11401 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:08.087796 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56142 (request call id 0) took 1328 ms. Trace:
I20260709 13:16:08.124035 12114 rpcz_store.cc:276] 0709 13:16:06.759756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.342243 (+582487us) service_pool.cc:224] Handling call
0709 13:16:08.087579 (+745336us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b540021dc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9913853 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:08.087707 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107729280}
I20260709 13:16:08.087797 14160 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:08.127210 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.128575 11545 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:08.089561 11398 raft_consensus.cc:2410] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:16:08.009809 12112 rpcz_store.cc:276] 0709 13:16:05.545423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.349555 (+1804132us) service_pool.cc:224] Handling call
0709 13:16:07.527531 (+177976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.528636 (+  1105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.903688 (+375052us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54005cf650. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:07.903907 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398335360}
W20260709 13:16:08.091801 14052 consensus_meta.cc:287] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.618s	user 0.006s	sys 0.004s
I20260709 13:16:08.130517 14052 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:08.093490 14170 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.001s
I20260709 13:16:08.095074 13990 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:08.131525 12113 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:08.131556 14170 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.135954 11468 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:08.141794 11396 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:08.095752 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:07.832240 14224 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:07.692022 14156 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:08.105304 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 68745.5us
I20260709 13:16:08.106606 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.144805 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.145488 11469 raft_consensus.cc:2393] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.145608 14224 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:08.146159 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.146621 11477 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.113749 11474 raft_consensus.cc:2368] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:08.147944 11474 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:08.149158 11474 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:08.150156 11474 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:08.058228 11470 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:08.157439 11470 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:08.146872 11469 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:08.159839 11551 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:08.160197 11545 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:08.160919 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.062494 14100 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.104445 14173 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:08.162451 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47033.1us
W20260709 13:16:08.189421 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 113951us
I20260709 13:16:08.044675 13950 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.166081 14204 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.166942 14183 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.168634 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.204277 11476 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:08.171983 14253 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:16:08.174862 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.206847 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.212926 11546 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:08.214958 14240 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.754s	user 0.002s	sys 0.001s
I20260709 13:16:08.205165 14241 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.199339 14213 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:16:08.221354 14213 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:08.221701 14213 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:08.191706 14216 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:08.224284 14216 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:08.225059 14216 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.226212 14216 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:08.183615 14160 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:08.232885 14160 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.235304 11550 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.236423 13950 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:08.198786 13879 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:08.186694 14219 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:16:08.237916 14219 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:08.238256 14219 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:08.241075 14173 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:08.195159 14218 consensus_meta.cc:287] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
W20260709 13:16:08.176419 14177 consensus_meta.cc:287] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.002s
W20260709 13:16:08.097739 14243 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
W20260709 13:16:08.177290 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 446123us
W20260709 13:16:08.181978 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 65256.2us
I20260709 13:16:08.264981 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.265851 11543 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:08.182925 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.058352 14237 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:08.193032 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49196.6us
W20260709 13:16:08.199002 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 293958us
I20260709 13:16:08.177112 12115 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:08.203250 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:08.284649 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:08.285665 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82483.9us
W20260709 13:16:08.207661 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.195827 14179 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:16:08.259364 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 52590.2us
I20260709 13:16:08.196383 14170 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:16:08.298530 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.268231 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:08.270053 11551 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:08.299077 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.103199 13936 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:16:08.305233 11401 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:16:08.291507 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.295110 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.267414 11469 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.292380 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.296299 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.297183 11441 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:08.297062 14064 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:08.308539 14162 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.222522 14213 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.311051 13950 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:08.311268 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.311897 14213 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:08.312083 11471 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.304723 14241 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:08.313525 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.315014 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.315659 11547 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:08.316521 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.317247 11547 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:08.259310 14003 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.318773 14003 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.320081 14236 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:08.288343 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:08.320925 14236 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.321288 14236 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:08.321647 11546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:16:08.322165 14236 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.308872 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:08.323158 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.324055 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.324877 11366 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.299392 11551 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:08.326562 14226 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.326706 11551 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.327268 13767 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:08.327739 11551 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:08.327934 14177 consensus_queue.cc:1175] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e9c2603c0bd4d25b713649b3890c46b" responder_term: 5 status { last_received { term: 5 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 5 index: 7 } }
I20260709 13:16:08.328639 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:08.324086 13873 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:08.329704 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:08.329968 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:08.318221 12116 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:16:08.293938 14114 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:08.330497 12116 pending_rounds.cc:85] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 6
I20260709 13:16:08.330797 12116 pending_rounds.cc:107] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:16:08.331092 12116 raft_consensus.cc:2889] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:08.332482 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:08.261709 14219 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.261087 14218 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.333832 14219 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:08.335176 11473 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:08.327131 14226 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:08.336890 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:08.295787 14156 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:08.335001 14218 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:08.337464 11544 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:08.310750 14162 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:08.337730 14156 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.339447 14187 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.339834 14156 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:08.341074 13767 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:08.341506 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.341570 13767 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:08.341764 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.342373 11477 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.342690 13767 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.344588 11474 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:16:08.345043 11476 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:08.299922 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.346122 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.343127 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.347345 11471 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:08.350044 11544 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:08.352389 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:08.353116 11397 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:08.354298 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:08.354494 11469 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:08.322736 14265 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:08.355013 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 57884.4us
I20260709 13:16:08.337103 14226 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.356817 13802 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:08.366348 13873 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:08.367010 14214 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.357225 14178 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:08.357545 14237 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.358613 12113 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:08.351699 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.361239 11545 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:08.355907 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.363301 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:08.373378 11469 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.380355 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.381230 11469 raft_consensus.cc:2368] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:08.382148 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.370522 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.384014 11471 raft_consensus.cc:2410] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:16:08.372634 11477 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:08.382972 11469 raft_consensus.cc:2368] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
W20260709 13:16:08.363747 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.364091 12119 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:08.364221 11367 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:08.362743 12116 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:08.365042 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:08.404269 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.404422 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.405367 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:08.368533 11547 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:16:08.405798 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 134230us
I20260709 13:16:08.369347 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:08.397238 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.393014 11398 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.406741 11469 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:08.407835 14102 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:08.408277 14102 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.408648 14102 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:08.408834 13873 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:08.409343 13873 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:08.411222 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47072.5us
I20260709 13:16:08.414254 13879 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.415838 14187 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:08.418380 13818 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.414887 13879 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:08.420370 13879 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.406651 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.423171 14123 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:08.423619 14123 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:08.424540 14123 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.425669 13936 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:08.425782 14123 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:08.427386 11514 leader_election.cc:336] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.559s (SENT)
I20260709 13:16:08.438043 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:08.428364 12123 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.427237 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.440456 11477 raft_consensus.cc:1217] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 13:16:08.441159 11477 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:08.441952 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:08.441951 11514 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.586s (SENT)
I20260709 13:16:08.443496 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:08.448433 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 80019.1us
I20260709 13:16:08.472769 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.477196 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:08.483613 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56476.5us
I20260709 13:16:08.486059 11365 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:08.370085 13802 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:08.438184 13984 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:08.527745 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.528688 11477 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
W20260709 13:16:08.530350 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44385.3us
I20260709 13:16:08.410437 13873 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.447115 14214 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:08.437896 14236 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.497076 13990 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:08.576964 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.471606 13902 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:08.471668 13956 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.544207 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.581103 11477 raft_consensus.cc:2368] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:08.541679 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.546443 14239 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:08.545997 14164 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.567576 14214 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:08.571437 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:08.497129 14237 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.572531 13936 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:08.578315 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.522430 13758 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:08.551653 14183 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:08.586529 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.621924 11546 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:08.622371 11546 raft_consensus.cc:614] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:08.572801 14265 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:16:08.539963 14171 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:08.448858 14162 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:16:08.603611 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 352) took 1313 ms. Trace:
I20260709 13:16:08.605698 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 304) took 1278 ms. Trace:
W20260709 13:16:08.611516 11514 leader_election.cc:336] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.842s (SENT)
I20260709 13:16:08.499364 14187 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:16:08.624904 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.629388 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:08.629045 11399 rpcz_store.cc:276] 0709 13:16:07.290472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.291106 (+   634us) service_pool.cc:224] Handling call
0709 13:16:07.291445 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:07.291789 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.291817 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:08.286786 (+994969us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:08.343782 (+ 56996us) log.cc:844] Serialized 52 byte log entry
0709 13:16:08.521582 (+177800us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:08.522913 (+  1331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.523015 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.589089 (+ 66074us) raft_consensus.cc:1685] finished
0709 13:16:08.589115 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.603507 (+ 14392us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1704,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":10032896,"thread_start_us":177282,"threads_started":1,"wal-append.queue_time_us":227199}
W20260709 13:16:08.670629 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 59166.6us
W20260709 13:16:08.671351 11514 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.856s (SENT)
W20260709 13:16:08.673981 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.541167 14027 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:16:08.629106 11542 rpcz_store.cc:276] 0709 13:16:07.326942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.327143 (+   201us) service_pool.cc:224] Handling call
0709 13:16:07.333628 (+  6485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.370146 (+ 36518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.605493 (+1235347us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b5400134050. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:08.605640 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13998848}
I20260709 13:16:08.620996 13936 consensus_queue.cc:1243] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:08.679307 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.570852 14219 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.622303 14190 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:08.689474 11514 leader_election.cc:336] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.907s (SENT)
I20260709 13:16:08.690089 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:08.692555 11514 leader_election.cc:336] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.912s (SENT)
W20260709 13:16:08.693688 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.695879 13802 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.623392 12115 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:08.697906 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 414) took 1144 ms. Trace:
I20260709 13:16:08.698254 12115 rpcz_store.cc:276] 0709 13:16:07.553093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.557869 (+  4776us) service_pool.cc:224] Handling call
0709 13:16:08.697844 (+1139975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.672902 14164 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.675149 14178 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:08.712122 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.714242 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.629673 13990 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:16:08.733891 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:08.764578 11514 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.999s (SENT)
I20260709 13:16:08.594431 13956 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.625994 14262 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:16:08.702212 14239 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:08.688230 14190 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:08.806216 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:08.806610 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 211698us
I20260709 13:16:08.821226 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:08.822499 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.822662 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1820) took 1443 ms. Trace:
I20260709 13:16:08.823100 11472 rpcz_store.cc:276] 0709 13:16:07.379578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.379713 (+   135us) service_pool.cc:224] Handling call
0709 13:16:07.380054 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.475195 (+ 95141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.822408 (+1347213us) spinlock_profiling.cc:243] Waited 94.3 ms on lock 0x7b540039a8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:08.822599 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216949632}
W20260709 13:16:08.917794 11366 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.3.62:34815, user_credentials={real_user=slave}} blocked reactor thread for 95857.3us
W20260709 13:16:08.917994 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 476137us
I20260709 13:16:08.797788 14213 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.820230 14190 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:08.948127 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:08.925204 14102 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:16:08.955237 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:08.955380 14102 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:08.957787 14269 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:16:08.961158 12123 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:16:08.967087 13818 consensus_meta.cc:287] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.003s
I20260709 13:16:08.967783 11402 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
W20260709 13:16:08.969707 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1700) took 1385 ms (client timeout 1500 ms). Trace:
W20260709 13:16:08.970120 11402 rpcz_store.cc:269] 0709 13:16:07.584332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.586659 (+  2327us) service_pool.cc:224] Handling call
0709 13:16:08.969445 (+1382786us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b54001266d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:08.969575 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204007680}
I20260709 13:16:08.971283 14191 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.972321 14102 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:08.976055 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1706) took 1239 ms (client timeout 1624 ms). Trace:
W20260709 13:16:08.976517 11396 rpcz_store.cc:269] 0709 13:16:07.737165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.007785 (+270620us) service_pool.cc:224] Handling call
0709 13:16:08.975875 (+968090us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400149bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:08.975992 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230056320}
I20260709 13:16:08.978029 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1822) took 1597 ms. Trace:
I20260709 13:16:08.978467 11468 rpcz_store.cc:276] 0709 13:16:07.380893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.381005 (+   112us) service_pool.cc:224] Handling call
0709 13:16:07.381256 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.136368 (+755112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.977773 (+841405us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540005ad54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:08.977966 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734887168}
I20260709 13:16:08.980585 11470 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:08.972010 14191 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:08.961095 14274 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.261s	user 0.000s	sys 0.000s
I20260709 13:16:08.988333 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.776407 14164 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:08.929378 14219 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:16:09.035032 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:08.935158 14213 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:09.078766 11514 leader_election.cc:336] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.524s (SENT)
I20260709 13:16:09.085086 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:09.008917 14084 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.108192 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:09.108922 11468 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.109866 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:09.020896 14262 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 13:16:09.114689 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.101578 13767 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:09.119984 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 301831us
I20260709 13:16:09.123903 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:09.117868 14216 consensus_meta.cc:287] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.889s	user 0.013s	sys 0.000s
I20260709 13:16:09.111435 14286 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.164382 14286 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:09.166285 14123 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.719s	user 0.003s	sys 0.006s
I20260709 13:16:09.172775 11469 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:09.153569 14191 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:09.156783 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1820) took 1927 ms. Trace:
I20260709 13:16:09.101300 14084 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:09.158835 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:09.161578 14213 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.167438 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:09.168107 11393 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:09.121660 14278 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:09.137599 14280 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:16:09.175580 11548 rpcz_store.cc:276] 0709 13:16:07.229012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.229177 (+   165us) service_pool.cc:224] Handling call
0709 13:16:07.247853 (+ 18676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.248315 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.248357 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:07.248931 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:16:09.156131 (+1907200us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b540003e920. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da98906b8 00007f8da991992b 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c
0709 13:16:09.156207 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:09.156273 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.156371 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:09.156426 (+    55us) raft_consensus.cc:1685] finished
0709 13:16:09.156442 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.156708 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2476,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7205,"spinlock_wait_cycles":121878784,"thread_start_us":862913,"threads_started":1,"wal-append.queue_time_us":719354,"wal-append.run_cpu_time_us":2133,"wal-append.run_wall_time_us":1157481}
I20260709 13:16:09.186757 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.205478 11472 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:09.205853 14281 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:16:09.207690 14279 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:16:09.192788 11476 raft_consensus.cc:2410] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:09.180361 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:09.211596 11469 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.212335 11475 pending_rounds.cc:85] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 2
I20260709 13:16:09.212625 11475 pending_rounds.cc:107] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:16:09.212853 11475 raft_consensus.cc:2889] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:09.212908 14249 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.204202 11399 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:09.213557 14249 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:09.215029 11471 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:09.215193 14303 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:09.216291 11471 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:09.216389 14303 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:09.216977 14252 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:09.217438 14252 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:09.217581 14302 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:09.211098 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1814) took 1834 ms. Trace:
I20260709 13:16:09.218619 11473 rpcz_store.cc:276] 0709 13:16:07.376768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.376952 (+   184us) service_pool.cc:224] Handling call
0709 13:16:07.414057 (+ 37105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.335604 (+921547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.210837 (+875233us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b540042a9d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:09.211036 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117954304}
I20260709 13:16:09.218902 11469 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:09.221251 11470 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:09.236284 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.218050 14302 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:09.520577 14084 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.520745 13984 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.521279 13984 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:09.521782 14283 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:16:09.523381 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1843) took 1190 ms. Trace:
I20260709 13:16:09.523752 11547 rpcz_store.cc:276] 0709 13:16:08.333023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.523841 (+190818us) service_pool.cc:224] Handling call
0709 13:16:08.524116 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.524360 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.524389 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.535285 (+ 10896us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.917406 (+382121us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:08.917448 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.917519 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:09.523102 (+605583us) raft_consensus.cc:1685] finished
0709 13:16:09.523134 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.523313 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":363116,"threads_started":1,"wal-append.queue_time_us":540000}
I20260709 13:16:09.525499 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1274) took 1234 ms. Trace:
I20260709 13:16:09.525943 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1846) took 1106 ms. Trace:
I20260709 13:16:09.525885 11398 rpcz_store.cc:276] 0709 13:16:08.291541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.291954 (+   413us) service_pool.cc:224] Handling call
0709 13:16:08.328616 (+ 36662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.406948 (+ 78332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.168728 (+761780us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b540019a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:09.525302 (+356574us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b540019a350. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:09.525439 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231316608}
I20260709 13:16:09.244856 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:09.529383 11393 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
W20260709 13:16:09.518724 12119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 455) took 1966 ms (client timeout 1871 ms). Trace:
W20260709 13:16:09.550177 12119 rpcz_store.cc:269] 0709 13:16:07.552555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.562654 (+ 10099us) service_pool.cc:224] Handling call
0709 13:16:09.518474 (+1955820us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54004c2554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:09.518632 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595073280}
I20260709 13:16:09.526685 11547 rpcz_store.cc:276] 0709 13:16:08.419142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.525031 (+1105889us) service_pool.cc:224] Handling call
0709 13:16:09.525338 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.525538 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.525575 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:09.525630 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:09.525678 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.525752 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.525875 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:09.573688 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:09.319061 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.559614 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1840) took 1227 ms. Trace:
I20260709 13:16:09.575450 11544 rpcz_store.cc:276] 0709 13:16:08.332381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.364281 (+ 31900us) service_pool.cc:224] Handling call
0709 13:16:08.364560 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.369065 (+  4505us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:08.369110 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.402495 (+ 33385us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.710590 (+308095us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:08.710642 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.710732 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:09.469293 (+758561us) raft_consensus.cc:1685] finished
0709 13:16:09.469327 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.559521 (+ 90194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3726,"raft.queue_time_us":3938,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":33824,"thread_start_us":183515,"threads_started":1,"wal-append.queue_time_us":507617}
W20260709 13:16:09.350124 12112 consensus_meta.cc:287] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.649s	user 0.006s	sys 0.001s
I20260709 13:16:09.452802 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:09.579581 12112 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:09.214797 14100 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.574751 11547 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
W20260709 13:16:09.580495 11514 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.135s (SENT)
I20260709 13:16:09.321077 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:09.482295 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.616243 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:09.616992 11400 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
W20260709 13:16:09.623760 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.519340 14286 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:09.519855 14164 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:09.407091 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.629475 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.638200 11469 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
W20260709 13:16:09.583709 14173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.010s	sys 0.022s
W20260709 13:16:09.396512 14240 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:16:09.644457 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:09.630700 11514 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.184s (SENT)
I20260709 13:16:09.645720 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:09.640664 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:09.243192 14248 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.646595 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1849) took 1207 ms. Trace:
I20260709 13:16:09.651597 11551 rpcz_store.cc:276] 0709 13:16:08.438953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.625025 (+1186072us) service_pool.cc:224] Handling call
0709 13:16:09.646007 (+ 20982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.646192 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.646220 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:09.646258 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:09.646299 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.646432 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.646541 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:09.656183 11514 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.193s (SENT)
I20260709 13:16:09.656682 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:09.657629 11514 leader_election.cc:336] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.205s (SENT)
W20260709 13:16:09.660589 14294 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.264s	user 0.000s	sys 0.000s
I20260709 13:16:09.661247 14301 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:09.598389 13956 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.666617 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1850) took 1214 ms. Trace:
I20260709 13:16:09.662122 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1841) took 1329 ms. Trace:
I20260709 13:16:09.667064 11551 rpcz_store.cc:276] 0709 13:16:08.452589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.652817 (+1200228us) service_pool.cc:224] Handling call
0709 13:16:09.666099 (+ 13282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.666290 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.666318 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:09.666358 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:09.666390 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.666445 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.666561 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:09.645350 14173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.010s	sys 0.022s
W20260709 13:16:09.668315 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 405882us
I20260709 13:16:09.667750 11545 rpcz_store.cc:276] 0709 13:16:08.332606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.392681 (+ 60075us) service_pool.cc:224] Handling call
0709 13:16:08.430572 (+ 37891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.674236 (+243664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.674266 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.676667 (+  2401us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.926825 (+250158us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:08.926872 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.926954 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:09.661872 (+734918us) raft_consensus.cc:1685] finished
0709 13:16:09.661898 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.662061 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47964,"raft.queue_time_us":123767,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":404738,"threads_started":2,"wal-append.queue_time_us":472985}
W20260709 13:16:09.671444 11514 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.225s (SENT)
I20260709 13:16:09.679148 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:09.218878 14278 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:09.686770 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.655485 14300 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:09.695967 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:09.701426 14180 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:09.701900 14180 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:09.702333 14180 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.703040 13873 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:09.703197 14302 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:09.704574 14302 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.710057 14249 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:09.719298 11514 leader_election.cc:336] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.257s (SENT)
I20260709 13:16:09.719830 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:09.721303 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.721993 11469 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:09.722864 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.725245 11469 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:09.728857 12119 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:09.316864 14252 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:09.733503 12123 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:09.734288 12117 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [5.8-6.9]
I20260709 13:16:09.733552 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:09.695241 13956 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:09.745131 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:09.746070 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 461) took 1477 ms. Trace:
I20260709 13:16:09.746497 12118 rpcz_store.cc:276] 0709 13:16:08.268445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.268554 (+   109us) service_pool.cc:224] Handling call
0709 13:16:09.744977 (+1476423us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:09.745999 (+  1022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13709056}
W20260709 13:16:09.757786 11475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.015s	sys 0.025s
W20260709 13:16:09.758266 11475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.015s	sys 0.025s
I20260709 13:16:09.756541 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:09.726266 14301 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:09.800410 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:09.801384 14084 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:09.801802 14084 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.802160 14084 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.807770 11543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:16:09.811945 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1206) took 1004 ms. Trace:
I20260709 13:16:09.812477 11476 rpcz_store.cc:276] 0709 13:16:08.807150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.807399 (+   249us) service_pool.cc:224] Handling call
0709 13:16:09.811641 (+1004242us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54004295d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:09.811850 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851819776}
I20260709 13:16:09.813868 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.816478 14084 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:09.819105 14084 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.820216 14239 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.820741 14239 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:09.821687 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.822854 14239 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.825246 14178 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:09.828122 14003 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.831724 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:09.832537 12118 raft_consensus.cc:2376] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5. Re-sending same reply.
I20260709 13:16:09.836633 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:09.837913 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:09.838893 14084 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.837718 14003 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:09.840186 14003 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.840623 14003 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.830497 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.839949 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:09.843397 11394 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.845014 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.853696 12118 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:09.856705 12119 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:09.847512 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:09.864828 11394 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:09.875934 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:09.872823 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.877445 11476 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:09.880594 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 124119us
I20260709 13:16:09.882648 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.883613 11476 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
W20260709 13:16:09.884449 11468 consensus_meta.cc:287] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.002s
I20260709 13:16:09.885246 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:09.885562 11468 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:09.886976 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:09.879976 11514 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.597s (SENT)
I20260709 13:16:09.771049 14252 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.729017 14262 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.135s
W20260709 13:16:09.923348 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.928990 14003 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:09.930900 14196 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:09.933354 14003 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:09.933662 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:09.934278 11394 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.923776 14189 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:09.936750 14189 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.937104 14189 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:09.937304 12122 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:16:09.937630 12122 pending_rounds.cc:85] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 7
I20260709 13:16:09.937844 12122 pending_rounds.cc:107] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:16:09.938083 12122 raft_consensus.cc:2889] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:09.923507 14084 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:09.938879 13984 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:09.939484 14084 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.939908 14084 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.929917 14164 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.930303 14303 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:09.794032 14298 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:16:09.939795 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:09.943710 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:09.930159 14237 consensus_meta.cc:287] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.309s	user 0.008s	sys 0.003s
W20260709 13:16:09.947299 11514 leader_election.cc:336] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.666s (SENT)
I20260709 13:16:09.949534 11394 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:09.917928 13902 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:10.058502 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.102962 14307 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:16:10.069002 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 405) took 1978 ms. Trace:
I20260709 13:16:10.012848 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:10.103842 11393 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:10.044808 14278 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:10.103556 11474 rpcz_store.cc:276] 0709 13:16:08.090809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.156743 (+ 65934us) service_pool.cc:224] Handling call
0709 13:16:08.344361 (+187618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.344942 (+   581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.068692 (+1723750us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540039bcd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:10.068918 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430595328}
W20260709 13:16:10.111660 12112 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 459) took 1843 ms (client timeout 2041 ms). Trace:
W20260709 13:16:10.112291 14305 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:16:10.116571 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 418) took 1814 ms. Trace:
I20260709 13:16:10.116990 12122 rpcz_store.cc:276] 0709 13:16:08.302105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.302253 (+   148us) service_pool.cc:224] Handling call
0709 13:16:08.327744 (+ 25491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.938498 (+1610754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.116326 (+177828us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54006c0554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:10.116511 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3700817152}
W20260709 13:16:10.119076 14310 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:16:10.121742 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 360) took 1677 ms. Trace:
I20260709 13:16:10.122208 11401 rpcz_store.cc:276] 0709 13:16:08.444761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.444923 (+   162us) service_pool.cc:224] Handling call
0709 13:16:09.168977 (+724054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.169186 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.169209 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.186745 (+ 17536us) log.cc:844] Serialized 26 byte log entry
0709 13:16:09.640281 (+453536us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:09.640316 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.640472 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.120262 (+479790us) raft_consensus.cc:1685] finished
0709 13:16:10.120286 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.121431 (+  1145us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b540019a350. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:10.121670 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651323648,"thread_start_us":453056,"threads_started":1,"wal-append.queue_time_us":686919}
I20260709 13:16:10.123559 11393 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:10.125978 14306 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:16:10.129328 14171 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:10.129738 14171 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:10.112133 12112 rpcz_store.cc:269] 0709 13:16:08.267878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.268004 (+   126us) service_pool.cc:224] Handling call
0709 13:16:10.111455 (+1843451us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400605c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:10.111586 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920660352}
I20260709 13:16:10.136899 11551 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:10.130745 14171 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:10.139951 14313 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:16:10.141876 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.143175 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.143358 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:10.143965 11471 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:10.144212 11549 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:10.145011 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.145589 11471 raft_consensus.cc:2376] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4. Re-sending same reply.
I20260709 13:16:10.146430 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.146952 11471 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:10.150410 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.065012 13984 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:09.798691 14204 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 13:16:09.981272 14262 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.980590 14164 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:09.873140 14249 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 13:16:09.940783 14219 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.030874 14236 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.455s	user 0.012s	sys 0.001s
I20260709 13:16:10.094811 11543 raft_consensus.cc:606] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:10.177171 11543 raft_consensus.cc:614] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:10.135504 13873 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:10.052693 14052 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:10.173487 14190 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.193s	user 0.006s	sys 0.005s
W20260709 13:16:10.090601 14309 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:16:10.155782 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:10.210083 11549 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:10.210454 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 325) took 1105 ms. Trace:
I20260709 13:16:10.211238 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:10.211138 11551 rpcz_store.cc:276] 0709 13:16:09.104806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.692551 (+587745us) service_pool.cc:224] Handling call
0709 13:16:09.692882 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.137382 (+444500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.210099 (+ 72717us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54001ec8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:10.210370 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020269440}
I20260709 13:16:10.212173 11549 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:10.213497 11551 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:10.213711 11549 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:10.011513 11476 raft_consensus.cc:2368] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 4. Current term is 5.
I20260709 13:16:10.215427 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.216342 11476 raft_consensus.cc:2393] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:10.217175 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.218142 11476 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:10.112152 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.102805 14224 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:10.039444 14296 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 13:16:10.170094 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.173167 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.176368 12114 consensus_meta.cc:287] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.583s	user 0.010s	sys 0.003s
I20260709 13:16:10.222744 11474 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:10.182924 11472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.054s	sys 0.065s
I20260709 13:16:10.191632 12116 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:16:10.225057 11472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.054s	sys 0.065s
W20260709 13:16:10.096058 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 152416us
I20260709 13:16:10.143476 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:10.101257 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:10.228283 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 281014us
W20260709 13:16:10.192404 14214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.009s	sys 0.009s
I20260709 13:16:10.173220 14171 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:10.206715 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.253305 11470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.006s	sys 0.014s
W20260709 13:16:10.253727 11470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.014s
I20260709 13:16:10.226162 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1843) took 1052 ms. Trace:
I20260709 13:16:10.205404 11546 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:16:10.232756 11514 leader_election.cc:336] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.711s (SENT)
W20260709 13:16:10.245641 11297 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 229) took 24993 ms (client timeout 15000 ms). Trace:
I20260709 13:16:10.248852 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.255280 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:10.254540 11472 rpcz_store.cc:276] 0709 13:16:09.173546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.206107 (+ 32561us) service_pool.cc:224] Handling call
0709 13:16:09.206447 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.206653 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:09.643821 (+437168us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.225718 (+581897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:10.225800 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:10.225867 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.225952 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.226096 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51794,"prepare.queue_time_us":716388,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":111,"thread_start_us":1018165,"threads_started":2}
W20260709 13:16:10.256592 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.257172 11514 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.518s (SENT)
I20260709 13:16:10.257673 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:10.255686 11476 raft_consensus.cc:2410] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:10.231460 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:10.258926 11543 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:10.189137 13873 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:10.272753 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.246841 14303 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:10.222498 14224 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:10.290679 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:10.291632 11544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.031s	sys 0.039s
W20260709 13:16:10.310668 11544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.031s	sys 0.039s
I20260709 13:16:10.247797 14237 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:10.310062 11476 raft_consensus.cc:2376] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5. Re-sending same reply.
I20260709 13:16:10.228642 12114 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:10.221552 14219 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:10.297025 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43947us
I20260709 13:16:10.271183 14146 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:10.265753 14286 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.228607 14190 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:10.289796 13802 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:10.188060 14262 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:10.289393 14248 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.002s
I20260709 13:16:10.278290 13873 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:10.264874 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.009s	sys 0.009s
I20260709 13:16:10.332504 11476 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:10.304796 14303 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:10.330689 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 410180us
I20260709 13:16:10.290024 14278 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.436811 11477 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:10.438932 11472 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:10.328971 11543 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
W20260709 13:16:10.343941 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.010s
W20260709 13:16:10.443713 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.003s	sys 0.010s
W20260709 13:16:10.444088 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 621710us
I20260709 13:16:10.446507 11397 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:10.447599 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.333108 14299 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:10.334633 11547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.000s	sys 0.024s
I20260709 13:16:10.315635 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 543) took 1387 ms. Trace:
W20260709 13:16:10.344338 11441 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.007s
W20260709 13:16:10.688639 11441 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.006s	sys 0.007s
W20260709 13:16:10.689014 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 867890us
W20260709 13:16:10.673938 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.041s	sys 0.052s
W20260709 13:16:10.690251 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.041s	sys 0.052s
W20260709 13:16:10.512851 14319 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:16:10.514617 13984 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:16:10.495029 14302 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.000s
I20260709 13:16:10.356748 14248 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:10.513543 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.701274 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:10.489082 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:10.489096 14308 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:16:10.702484 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:10.707994 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.680946 12086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.028s	sys 0.004s
I20260709 13:16:10.356930 14286 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:10.511829 14315 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:16:10.720726 14178 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:10.721161 14178 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.721212 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1708) took 2890 ms. Trace:
I20260709 13:16:10.721638 14178 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.721666 11395 rpcz_store.cc:276] 0709 13:16:07.830837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.121598 (+290761us) service_pool.cc:224] Handling call
0709 13:16:08.121857 (+   259us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:08.200358 (+ 78501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.200405 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:09.616218 (+1415813us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:09.625243 (+  9025us) log.cc:844] Serialized 52 byte log entry
0709 13:16:10.029026 (+403783us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:10.051898 (+ 22872us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.052295 (+   397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.052404 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.720917 (+668513us) raft_consensus.cc:1685] finished
0709 13:16:10.721021 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.721155 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111506,"prepare.queue_time_us":23344,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":88174,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":22207,"spinlock_wait_cycles":8801536,"thread_start_us":301916,"threads_started":1,"wal-append.queue_time_us":684054}
I20260709 13:16:10.728567 14249 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:10.729065 14249 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:10.729866 14249 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:10.730551 14252 consensus_meta.cc:287] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.004s
I20260709 13:16:10.700445 13802 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:10.731683 13802 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.732293 14178 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:10.733662 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1278) took 2111 ms. Trace:
I20260709 13:16:10.734277 13802 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:10.734607 14178 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:10.685492 11471 consensus_meta.cc:287] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.513s	user 0.004s	sys 0.007s
I20260709 13:16:10.735189 11471 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
W20260709 13:16:10.447502 14180 consensus_meta.cc:287] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.006s
I20260709 13:16:10.695075 14226 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:10.736040 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.697919 14237 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:16:10.255934 11297 rpcz_store.cc:269] 0709 13:15:45.252453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.252682 (+   229us) service_pool.cc:224] Handling call
0709 13:15:46.717058 (+1464376us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:10.245514 (+23528456us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:05.230294 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:05.231048 (+   754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:05.231122 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:16:05.231141 (+    19us) tablet.cc:662] Decoding operations
0709 13:16:05.260006 (+ 28865us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:05.260098 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:05.260128 (+    30us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:16:05.564543 (+304415us) tablet.cc:701] Row locks acquired
0709 13:16:05.564594 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:16:05.564792 (+   198us) write_op.cc:270] Start()
0709 13:16:05.564952 (+   160us) write_op.cc:276] Timestamp: P: 1783602965564715 usec, L: 0
0709 13:16:05.564992 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:16:05.721253 (+156261us) log.cc:844] Serialized 9666 byte log entry
0709 13:16:06.119431 (+398178us) op_driver.cc:464] REPLICATION: finished
0709 13:16:06.305539 (+186108us) write_op.cc:301] APPLY: starting
0709 13:16:06.305642 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:06.306140 (+   498us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:06.306293 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:06.439847 (+133554us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:06.440253 (+   406us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0709 13:16:06.440300 (+    47us) write_op.cc:312] APPLY: finished
0709 13:16:06.440819 (+   519us) log.cc:844] Serialized 170 byte log entry
0709 13:16:06.461196 (+ 20377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:06.481916 (+ 20720us) write_op.cc:454] Released schema lock
0709 13:16:06.497485 (+ 15569us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:06.519992 (+ 22507us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:06.520024 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":224737,"reported_tablets":39,"spinlock_wait_cycles":1583563008,"child_traces":[["op",{"apply.queue_time_us":179317,"apply.run_cpu_time_us":7029,"apply.run_wall_time_us":214414,"num_ops":25,"prepare.queue_time_us":790490,"prepare.run_cpu_time_us":20782,"prepare.run_wall_time_us":584907,"raft.queue_time_us":78851,"raft.run_cpu_time_us":22851,"raft.run_wall_time_us":62742,"replication_time_us":554222,"thread_start_us":672693,"threads_started":4,"wal-append.queue_time_us":148537,"wal-append.run_cpu_time_us":18456,"wal-append.run_wall_time_us":1556369}]]}
I20260709 13:16:10.736539 14226 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:10.737560 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1832) took 2526 ms. Trace:
W20260709 13:16:10.709702 12086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.028s	sys 0.004s
I20260709 13:16:10.480777 14299 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:10.712843 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.716967 14239 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:10.719923 14286 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:10.739137 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 326) took 1407 ms. Trace:
I20260709 13:16:10.739334 14299 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.739967 14214 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:10.738023 11550 rpcz_store.cc:276] 0709 13:16:08.245661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.260074 (+ 14413us) service_pool.cc:224] Handling call
0709 13:16:08.285614 (+ 25540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.338772 (+ 53158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.338805 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.346090 (+  7285us) log.cc:844] Serialized 26 byte log entry
0709 13:16:09.625404 (+1279314us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b1800454420. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:10.093666 (+468262us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:10.093766 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.093874 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.737271 (+643397us) raft_consensus.cc:1685] finished
0709 13:16:10.737292 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.737500 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22059,"raft.queue_time_us":16838,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":4629,"spinlock_wait_cycles":5924736,"thread_start_us":468021,"threads_started":1,"wal-append.queue_time_us":1944636}
I20260709 13:16:10.741214 14214 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:10.447357 14278 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:10.738628 12114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 460) took 2470 ms (client timeout 1823 ms). Trace:
W20260709 13:16:10.743474 12114 rpcz_store.cc:269] 0709 13:16:08.268169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.294116 (+ 25947us) service_pool.cc:224] Handling call
0709 13:16:10.738428 (+2444312us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54004c1b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:10.738536 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559828352}
I20260709 13:16:10.711289 14303 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:10.706521 14224 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.746835 14224 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.735942 14180 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:10.749032 14180 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:10.697297 14160 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.749737 14160 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.736903 11471 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:10.716336 14146 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:10.756383 11545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.025s	sys 0.037s
W20260709 13:16:10.758086 11545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.025s	sys 0.037s
I20260709 13:16:10.734052 11397 rpcz_store.cc:276] 0709 13:16:08.622283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.622474 (+   191us) service_pool.cc:224] Handling call
0709 13:16:08.622824 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.447038 (+1824214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.733437 (+286399us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54003a7ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:10.733606 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4193531648}
W20260709 13:16:10.754681 11514 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.992s (SENT)
I20260709 13:16:10.759465 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:10.685037 11547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.003s	sys 0.057s
I20260709 13:16:10.744002 14278 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:10.763970 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:10.738797 14226 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:10.770912 11514 leader_election.cc:336] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.042s (SENT)
I20260709 13:16:10.770901 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.771569 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1203) took 2178 ms. Trace:
W20260709 13:16:10.777257 11514 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.070s (SENT)
I20260709 13:16:10.793033 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:10.795250 12116 raft_consensus.cc:2393] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:10.792423 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.796509 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 324) took 1694 ms. Trace:
I20260709 13:16:10.796789 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.773712 14237 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:10.759964 14249 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:10.797843 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.797076 11545 rpcz_store.cc:276] 0709 13:16:09.102603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.674503 (+571900us) service_pool.cc:224] Handling call
0709 13:16:09.674766 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.674978 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:10.094111 (+419133us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.796214 (+702103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:10.796261 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:10.796303 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.796363 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.796462 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35416,"prepare.queue_time_us":664034,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":1064290,"threads_started":2}
I20260709 13:16:10.758163 13984 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:10.794020 11477 rpcz_store.cc:276] 0709 13:16:08.592905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.593056 (+   151us) service_pool.cc:224] Handling call
0709 13:16:08.593460 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.437329 (+1843869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.771313 (+333984us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400429fd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:10.771497 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4238806912}
I20260709 13:16:10.795755 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:10.794902 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:10.799876 11468 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.025s	sys 0.016s
W20260709 13:16:10.799963 11542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.021s	sys 0.029s
I20260709 13:16:10.801887 13802 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.790442 14286 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.774155 14239 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:10.790652 14003 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.824175 11473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.037s	sys 0.040s
W20260709 13:16:10.831696 11542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.021s	sys 0.029s
W20260709 13:16:10.880795 11473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.038s	sys 0.040s
I20260709 13:16:10.770159 14278 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:10.765765 14219 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:10.688378 12115 rpcz_store.cc:276] 0709 13:16:08.921990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.934363 (+ 12373us) service_pool.cc:224] Handling call
0709 13:16:10.143294 (+1208931us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:10.309464 (+166170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13228928}
I20260709 13:16:10.773782 14160 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:10.739530 11548 rpcz_store.cc:276] 0709 13:16:09.331967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.706785 (+374818us) service_pool.cc:224] Handling call
0709 13:16:09.707218 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.707660 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.707699 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.708287 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.094621 (+386334us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:10.094672 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.095274 (+   602us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.738860 (+643586us) raft_consensus.cc:1685] finished
0709 13:16:10.738878 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.739080 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":907,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":385953,"threads_started":1,"wal-append.queue_time_us":624950}
I20260709 13:16:10.767899 14299 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.799886 14100 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.801295 13902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.013s	sys 0.023s
I20260709 13:16:10.804005 13984 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:10.843067 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.021s	sys 0.016s
W20260709 13:16:10.937507 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.021s	sys 0.017s
W20260709 13:16:10.937932 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 813234us
I20260709 13:16:10.816636 14242 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:10.818403 14164 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:10.840987 14237 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.884497 14278 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:10.888478 14327 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:16:10.889094 14262 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:10.885687 14003 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.856616 14178 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:10.893049 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1848) took 2463 ms. Trace:
I20260709 13:16:10.943145 14100 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.957006 11547 rpcz_store.cc:276] 0709 13:16:08.429705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.599952 (+1170247us) service_pool.cc:224] Handling call
0709 13:16:09.600291 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.600809 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.600853 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.601509 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.760368 (+1158859us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:10.760403 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.760472 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.892768 (+132296us) raft_consensus.cc:1685] finished
0709 13:16:10.892795 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.892978 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2680,"raft.queue_time_us":115,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":13674,"thread_start_us":1094219,"threads_started":1,"wal-append.queue_time_us":1202313}
W20260709 13:16:10.893313 14316 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:16:10.943938 14160 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:10.943552 14242 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:10.914413 14171 consensus_meta.cc:287] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.649s	user 0.005s	sys 0.002s
I20260709 13:16:10.881675 14286 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.929123 14330 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.930474 14084 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.990s	user 0.007s	sys 0.004s
W20260709 13:16:10.804019 11468 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.025s	sys 0.016s
W20260709 13:16:10.847146 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.024s	sys 0.062s
W20260709 13:16:10.969383 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.024s	sys 0.062s
W20260709 13:16:10.969717 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 697836us
W20260709 13:16:10.961263 12084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.019s	sys 0.049s
W20260709 13:16:10.971276 12084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.019s	sys 0.049s
W20260709 13:16:10.937110 13902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.013s	sys 0.023s
W20260709 13:16:10.952543 14308 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:16:10.950824 14312 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.264s	user 0.001s	sys 0.001s
W20260709 13:16:10.960451 12119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.003s	sys 0.012s
W20260709 13:16:10.954466 14189 consensus_meta.cc:287] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.017s	user 0.010s	sys 0.006s
I20260709 13:16:10.898092 14190 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:10.961108 13873 consensus_meta.cc:287] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
W20260709 13:16:10.960839 11368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.001s	sys 0.016s
I20260709 13:16:10.966588 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.945173 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.986524 11471 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:10.982268 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1849) took 1350 ms. Trace:
I20260709 13:16:10.987505 11470 rpcz_store.cc:276] 0709 13:16:09.631637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.631813 (+   176us) service_pool.cc:224] Handling call
0709 13:16:09.636205 (+  4392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.663326 (+ 27121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.663359 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.698043 (+ 34684us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.254131 (+556088us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:10.254204 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.254315 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.982071 (+727756us) raft_consensus.cc:1685] finished
0709 13:16:10.982089 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.982214 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22365,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"thread_start_us":546299,"threads_started":1,"wal-append.queue_time_us":1129522}
I20260709 13:16:10.989713 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:10.982600 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1844) took 1808 ms. Trace:
W20260709 13:16:10.992700 11368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.001s	sys 0.016s
I20260709 13:16:10.966573 14242 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:10.990454 11475 rpcz_store.cc:276] 0709 13:16:09.173851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.211847 (+ 37996us) service_pool.cc:224] Handling call
0709 13:16:09.212145 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:09.213483 (+  1338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.213508 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:09.214031 (+   523us) log.cc:844] Serialized 52 byte log entry
0709 13:16:09.758732 (+544701us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:09.758791 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.758889 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.982340 (+1223451us) raft_consensus.cc:1685] finished
0709 13:16:10.982365 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.982532 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":303,"raft.queue_time_us":5619,"raft.run_cpu_time_us":8173,"raft.run_wall_time_us":169252,"thread_start_us":544251,"threads_started":1,"wal-append.queue_time_us":1232686}
I20260709 13:16:11.001624 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1855) took 1325 ms. Trace:
I20260709 13:16:10.983245 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1845) took 2564 ms. Trace:
I20260709 13:16:10.995348 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1847) took 2566 ms. Trace:
I20260709 13:16:10.962337 14160 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:10.979825 14236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.002s	sys 0.010s
W20260709 13:16:10.983755 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 288087us
I20260709 13:16:10.974781 13902 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:11.000451 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:11.059417 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59057us
I20260709 13:16:11.058585 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.060235 11441 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:11.060602 11475 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:11.061050 11441 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:11.003221 11544 rpcz_store.cc:276] 0709 13:16:08.429384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.577782 (+1148398us) service_pool.cc:224] Handling call
0709 13:16:09.578213 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.578515 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.578556 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.579145 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.311234 (+732089us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:10.311284 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.311357 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.995116 (+683759us) raft_consensus.cc:1685] finished
0709 13:16:10.995140 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.995288 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":657524,"threads_started":1,"wal-append.queue_time_us":1136210}
I20260709 13:16:11.002213 11469 rpcz_store.cc:276] 0709 13:16:09.676118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.726859 (+ 50741us) service_pool.cc:224] Handling call
0709 13:16:09.727176 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.736462 (+  9286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.736492 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.756849 (+ 20357us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.192403 (+435554us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:10.192442 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.192502 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:11.001349 (+808847us) raft_consensus.cc:1685] finished
0709 13:16:11.001377 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.001550 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1418,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":395687,"threads_started":1,"wal-append.queue_time_us":727704}
I20260709 13:16:10.951892 14237 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:10.983229 12119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.004s	sys 0.012s
I20260709 13:16:10.981045 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 544) took 2058 ms. Trace:
I20260709 13:16:11.003100 11542 rpcz_store.cc:276] 0709 13:16:08.418732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.810220 (+391488us) service_pool.cc:224] Handling call
0709 13:16:08.916374 (+106154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.916799 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.916829 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.208137 (+291308us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b480002d9d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:10.127577 (+919440us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b2c000a0b68. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:10.128064 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:16:10.880950 (+752886us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:10.880989 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.881305 (+   316us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.983004 (+101699us) raft_consensus.cc:1685] finished
0709 13:16:10.983027 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.983182 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13861,"raft.run_cpu_time_us":2786,"raft.run_wall_time_us":273168,"spinlock_wait_cycles":55249536,"thread_start_us":752510,"threads_started":1,"wal-append.queue_time_us":852356}
I20260709 13:16:11.009276 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:10.995532 13879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.036s
I20260709 13:16:11.010841 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.012926 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:10.974524 14084 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.025964 14003 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:10.975811 14330 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:10.981766 14100 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:11.034070 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.106688 14330 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.039311 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:11.040958 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:11.107493 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:11.108210 12115 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
W20260709 13:16:11.108520 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67642.9us
W20260709 13:16:11.041389 11474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.009s	sys 0.045s
W20260709 13:16:11.111227 11474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.009s	sys 0.045s
I20260709 13:16:11.112488 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:11.113379 12114 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:10.984294 14189 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:11.080657 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.115700 14330 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.116173 14330 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.116638 14189 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.117388 14100 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:11.117913 14100 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:11.118170 14330 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:11.033627 14190 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.984638 14239 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.034159 14171 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:11.124991 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.008s	sys 0.044s
I20260709 13:16:11.065963 11547 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:11.100035 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:11.038470 14334 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.995282 14278 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:11.008752 14236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.003s	sys 0.010s
I20260709 13:16:11.110737 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:11.119596 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.121310 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.133517 11475 raft_consensus.cc:2410] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:11.134335 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:11.125376 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.008s	sys 0.044s
I20260709 13:16:11.125859 12116 raft_consensus.cc:2376] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9. Re-sending same reply.
I20260709 13:16:11.126637 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:11.135811 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 796491us
I20260709 13:16:11.136065 11472 raft_consensus.cc:2376] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
I20260709 13:16:11.136929 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.137435 11472 raft_consensus.cc:2376] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
I20260709 13:16:11.099291 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:11.138137 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 98898.7us
W20260709 13:16:11.139249 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.141239 12112 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:11.138213 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.002731 13873 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.128907 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.130538 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.031862 13984 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.103269 14173 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:16:11.103127 14164 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:16:11.097644 14331 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:16:11.125355 14296 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:16:11.118868 14100 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.101266 14160 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.232275 11402 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:11.423141 11544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
W20260709 13:16:11.424315 11473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.023s	sys 0.077s
I20260709 13:16:11.574007 11544 raft_consensus.cc:606] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:11.574398 11473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.023s	sys 0.088s
I20260709 13:16:11.574534 11544 raft_consensus.cc:614] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:11.574841 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 464187us
I20260709 13:16:11.575217 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1847) took 2400 ms. Trace:
I20260709 13:16:11.151786 11400 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:11.575743 11473 rpcz_store.cc:276] 0709 13:16:09.174833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.220857 (+ 46024us) service_pool.cc:224] Handling call
0709 13:16:09.321666 (+100809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.321951 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:09.776163 (+454212us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.881349 (+1105186us) log.cc:844] Serialized 14 byte log entry
0709 13:16:11.574860 (+693511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.574909 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:11.574950 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.575069 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.575167 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":820006,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2290,"raft.queue_time_us":152328,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7923,"thread_start_us":2153702,"threads_started":4,"wal-append.queue_time_us":1014822}
I20260709 13:16:11.576905 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:11.146384 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.588743 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.152696 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:11.593322 11400 raft_consensus.cc:2410] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:16:11.152297 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:11.163357 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.578847 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:11.579165 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.599290 11476 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:11.444777 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:11.600049 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.600293 11399 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
W20260709 13:16:11.600602 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.579447 11469 raft_consensus.cc:2410] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:11.185165 11398 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:11.579582 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:11.560493 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 379917us
I20260709 13:16:11.170053 11395 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:11.168148 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.605055 11471 raft_consensus.cc:2410] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:11.127586 14262 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:11.063125 12113 rpcz_store.cc:276] 0709 13:16:08.922397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.926043 (+  3646us) service_pool.cc:224] Handling call
0709 13:16:08.986774 (+ 60731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.084305 (+ 97531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.085631 (+  1326us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.151390 (+ 65759us) log.cc:844] Serialized 26 byte log entry
0709 13:16:09.633688 (+482298us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:10.019047 (+385359us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.041374 (+ 22327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.046078 (+  4704us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.980775 (+934697us) raft_consensus.cc:1685] finished
0709 13:16:10.980799 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.980963 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21525,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":10848,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6324,"thread_start_us":739111,"threads_started":2,"wal-append.queue_time_us":985538}
I20260709 13:16:11.602558 11472 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:11.613708 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.615751 11470 raft_consensus.cc:2393] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:11.252547 14052 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.008s	sys 0.016s
I20260709 13:16:11.619935 11470 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:16:11.621240 11470 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:11.621548 11470 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:11.443030 12115 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:16:11.184552 14252 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.422572 14178 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.146040 14027 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:11.147166 14257 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.181617 14219 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:11.100723 14340 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.525s
W20260709 13:16:11.633927 14052 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.014s	sys 0.030s
I20260709 13:16:11.128139 14190 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.531072 14286 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.440917 14329 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.422981 14336 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:11.626575 11472 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:16:11.381888 14338 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.179479 14330 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:11.617218 13818 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:16:11.219208 14187 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:11.624286 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:11.219210 14341 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:11.650422 11473 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:11.704957 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:11.652695 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.705948 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.676102 14160 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:11.604297 14278 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:11.381708 14180 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.638147 11441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.005s
I20260709 13:16:11.622483 11470 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:11.731652 11441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.006s	sys 0.005s
I20260709 13:16:11.674804 11471 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:11.732445 11471 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:11.614764 14262 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.402743 14349 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:11.686908 14084 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:11.132485 14334 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.634611 13984 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:11.455235 14343 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.214s
I20260709 13:16:11.694154 14361 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:11.691066 13802 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.694725 11544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:46030
I20260709 13:16:11.796666 11544 raft_consensus.cc:606] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:11.797067 11544 raft_consensus.cc:614] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:11.644619 14347 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.130434 14171 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:11.698351 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.803750 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.654376 14178 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.808218 14347 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.809394 14171 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.732599 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:11.454851 13902 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.692823 14219 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:11.781045 14262 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:11.778225 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:11.696142 14286 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:11.780936 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.736362 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 136511us
I20260709 13:16:11.805083 11471 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.653599 14252 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.685611 11476 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:11.786716 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.703064 14338 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.674525 14027 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.687198 14249 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:11.809491 14178 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:11.706468 11472 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:11.847831 11471 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:11.848188 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.682564 14257 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:11.731542 11468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.023s	sys 0.058s
W20260709 13:16:11.849409 11468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.024s	sys 0.059s
I20260709 13:16:11.828122 11473 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:11.830022 11402 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:11.778853 14353 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:11.845721 11514 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.742s (SENT)
W20260709 13:16:11.851008 14346 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 13:16:11.851358 14342 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:16:11.852629 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 549) took 1912 ms. Trace:
I20260709 13:16:11.853044 12119 rpcz_store.cc:276] 0709 13:16:09.940404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.940602 (+   198us) service_pool.cc:224] Handling call
0709 13:16:09.982641 (+ 42039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:10.088576 (+105935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:10.088619 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:10.203819 (+115200us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.122837 (+919018us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:11.122897 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.123002 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:11.852394 (+729392us) raft_consensus.cc:1685] finished
0709 13:16:11.852415 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.852564 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53217,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":102,"thread_start_us":766607,"threads_started":1,"wal-append.queue_time_us":954528}
I20260709 13:16:11.855093 14146 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:11.855288 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1859) took 1520 ms. Trace:
I20260709 13:16:11.855397 14358 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.855659 14146 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.856086 14358 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.856705 14338 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.857347 14257 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.858930 14146 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.859433 14146 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.859856 14146 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.860213 14252 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.862112 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:11.843015 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:11.855669 11474 rpcz_store.cc:276] 0709 13:16:10.334583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.335363 (+   780us) service_pool.cc:224] Handling call
0709 13:16:10.439414 (+104051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:10.439765 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:10.439791 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:10.442223 (+  2432us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.111724 (+669501us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:11.111787 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.111890 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:11.855054 (+743164us) raft_consensus.cc:1685] finished
0709 13:16:11.855087 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.855226 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":749,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":34774,"thread_start_us":669102,"threads_started":1,"wal-append.queue_time_us":1013956}
I20260709 13:16:11.862807 11476 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:11.864595 11396 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:11.702952 14330 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.866685 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 414) took 2122 ms. Trace:
I20260709 13:16:11.869618 11473 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:11.884262 11468 rpcz_store.cc:276] 0709 13:16:09.743997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.889603 (+145606us) service_pool.cc:224] Handling call
0709 13:16:09.889930 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.890171 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:10.968915 (+1078744us) log.cc:844] Serialized 14 byte log entry
0709 13:16:11.866280 (+897365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.866362 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:11.866424 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.866520 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.866634 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52842,"prepare.queue_time_us":155910,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"thread_start_us":1866680,"threads_started":2}
I20260709 13:16:11.758011 14180 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:11.864928 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:16:11.756923 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 103947us
I20260709 13:16:11.858525 14358 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.701560 14336 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.912621 14330 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:11.911846 11298 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 6, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 6 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:11.885011 14345 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 13:16:11.838668 14340 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.804025 13802 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.849133 14187 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.699555 14329 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.076848 14369 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.003773 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 153140us
W20260709 13:16:12.139441 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.138806 11298 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:12.096691 14190 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.991554 14372 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.889086 14353 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:11.966457 14224 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.219s	user 0.011s	sys 0.002s
I20260709 13:16:11.901628 13873 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:11.843899 14286 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:11.953955 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 221463us
I20260709 13:16:12.153793 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:12.154066 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:12.099229 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 173827us
W20260709 13:16:12.155128 11514 leader_election.cc:336] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.766s (SENT)
I20260709 13:16:11.700712 14352 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.228s
W20260709 13:16:11.988353 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.863112 14249 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.155592 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:11.861109 14219 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:11.849896 14361 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:12.027056 14330 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:11.734951 14341 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:12.159399 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:12.106273 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 412162us
I20260709 13:16:12.141191 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:12.084579 14367 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:12.145764 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:12.306021 11544 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
W20260709 13:16:12.148631 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.105803 14347 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:12.108106 14349 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:11.962522 11471 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:16:12.308462 11471 pending_rounds.cc:85] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 3
I20260709 13:16:12.308779 11471 pending_rounds.cc:107] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:12.309022 11471 raft_consensus.cc:2889] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:12.156315 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.156695 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:12.142328 11298 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 2 to 4, term changed from 3 to 5, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.155382 11365 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:12.142102 14187 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.319329 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1234) took 1520 ms. Trace:
I20260709 13:16:12.319754 11471 rpcz_store.cc:276] 0709 13:16:10.798950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.617400 (+818450us) service_pool.cc:224] Handling call
0709 13:16:11.649292 (+ 31892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:12.318758 (+669466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:12.319269 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:12.154359 14146 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:12.334002 12116 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:12.310379 11545 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:12.303570 13984 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.197234 14242 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.185s	user 0.010s	sys 0.001s
W20260709 13:16:12.192716 14355 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.086s	user 0.003s	sys 0.000s
I20260709 13:16:12.219252 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:12.183147 14348 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.218725 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63628.7us
I20260709 13:16:12.207856 14341 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:12.138851 14340 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.208967 14084 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:12.204243 14383 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:12.187752 14356 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:12.192950 14330 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.194286 14361 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.196261 13802 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:12.199314 14286 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:12.196949 14365 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
W20260709 13:16:12.154129 14299 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.217s	user 0.010s	sys 0.003s
I20260709 13:16:12.151244 14358 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.311841 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 156526us
I20260709 13:16:12.520951 14358 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.521346 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:12.523984 14360 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:12.524670 14360 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:12.308001 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:12.192086 14027 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:12.332227 14371 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:12.525750 14360 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.526407 14371 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:12.527345 14371 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.392946 14348 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.530869 14224 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:12.193447 14249 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:12.531646 14262 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.000s
I20260709 13:16:12.532187 14262 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.387665 14242 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.533643 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.534467 11474 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:12.391180 14180 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:12.536702 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:12.537456 11473 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:12.389991 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1230) took 1680 ms. Trace:
I20260709 13:16:12.310496 14334 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:12.538021 11475 rpcz_store.cc:276] 0709 13:16:10.709682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.589902 (+880220us) service_pool.cc:224] Handling call
0709 13:16:11.590265 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.590512 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:12.137477 (+546965us) log.cc:844] Serialized 14 byte log entry
0709 13:16:12.389600 (+252123us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:12.389669 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:12.389729 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:12.389807 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:12.389936 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":792753,"raft.queue_time_us":89,"thread_start_us":744229,"threads_started":2}
I20260709 13:16:12.540947 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.541644 11475 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:12.389313 14146 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:12.542728 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.544797 14146 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.308499 14178 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:12.381886 11440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.008s	sys 0.005s
I20260709 13:16:12.146629 14369 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.340669 14347 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.339052 14252 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.347245 13873 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:12.447057 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.152628 14372 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.321949 14367 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.507951 11298 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 2 to 3, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:12.382397 13984 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.517081 14239 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:12.710096 11440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.010s	sys 0.008s
W20260709 13:16:12.692543 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.542734 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.545264 11474 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 3.
W20260709 13:16:12.710589 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 488488us
I20260709 13:16:12.595532 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:12.740492 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:12.741780 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:12.742082 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.715355 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:12.750041 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 154630us
I20260709 13:16:12.714114 14190 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.772887 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:12.747613 11514 leader_election.cc:336] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.508s (SENT)
W20260709 13:16:12.815431 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:12.790491 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75228.2us
W20260709 13:16:12.791703 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47106.9us
W20260709 13:16:12.815776 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 72881.4us
W20260709 13:16:12.829468 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89050.2us
W20260709 13:16:12.827513 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.830230 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:12.830366 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:12.829064 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:12.740247 14347 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.830281 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:12.505831 14362 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.749737 13802 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:12.854707 12123 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:12.832458 11514 leader_election.cc:336] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.974s (SENT)
W20260709 13:16:12.688475 14160 consensus_meta.cc:287] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.950s	user 0.007s	sys 0.004s
W20260709 13:16:12.546676 14363 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
I20260709 13:16:12.804708 14052 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:12.739346 14336 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:12.722385 11473 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:12.872299 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.873286 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.873901 11473 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:12.742878 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1862) took 2401 ms. Trace:
I20260709 13:16:12.861780 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:12.740010 14226 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.862910 11514 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.012s (SENT)
I20260709 13:16:12.711050 13873 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.864055 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:12.839581 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:12.551762 14387 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:16:12.867499 14347 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:12.713991 14341 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.714048 14299 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.806037 14353 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:12.546967 14383 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:12.876296 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:12.504580 14352 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.831213 14376 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:12.909500 12117 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:12.805008 14286 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:12.752756 14371 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:12.790305 14224 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:12.556418 14340 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:12.929488 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66604.8us
W20260709 13:16:12.941942 11514 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.015s (SENT)
I20260709 13:16:12.870568 13802 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.942548 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:12.944010 11514 leader_election.cc:336] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.040s (SENT)
I20260709 13:16:12.944548 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:12.871011 14052 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.885253 12115 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:16:12.392570 14349 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.875682 14377 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:12.885035 14242 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:12.854626 14190 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.548709 14187 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:12.915135 13873 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:12.940325 14353 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:12.508973 14356 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.740283 14252 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.957192 14178 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:12.929996 14329 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:12.795630 14239 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.931069 14299 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.941012 14286 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.903443 14226 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:12.865231 14362 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.952394 14336 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.930152 14376 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.930866 14341 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.025247 14356 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:13.029918 14330 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.027133 14386 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.941346 14352 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:12.743775 11367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.3:46781 timed out after 1.595s (SENT) [suppressed 34 similar messages]
I20260709 13:16:12.878697 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:12.744036 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1863) took 1604 ms. Trace:
W20260709 13:16:12.946791 11514 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.043s (SENT)
I20260709 13:16:12.874295 11477 rpcz_store.cc:276] 0709 13:16:10.341238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.807783 (+466545us) service_pool.cc:224] Handling call
0709 13:16:10.808186 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:10.839663 (+ 31477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:10.839698 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:10.841759 (+  2061us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.252832 (+411073us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:11.252889 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.252989 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:12.535471 (+1282482us) raft_consensus.cc:1685] finished
0709 13:16:12.535498 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:12.742820 (+207322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173877,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":23851,"thread_start_us":410578,"threads_started":1,"wal-append.queue_time_us":1187482}
I20260709 13:16:12.836436 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1867) took 1647 ms. Trace:
I20260709 13:16:12.874806 11471 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:16:13.027089 13984 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:12.545929 14084 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:12.956883 14052 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:13.035269 14286 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:13.048446 14372 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.047031 14369 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.037221 14336 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:13.205760 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:13.181463 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.171322 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:13.356557 14376 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:13.035079 14252 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.191669 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1287) took 2055 ms. Trace:
W20260709 13:16:13.363210 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.365060 14367 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.367453 14336 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.371150 14361 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.372146 14361 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.375746 14190 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.377107 11470 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:13.341219 14343 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:13.379866 14343 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:13.381048 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.389211 14239 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.390089 14249 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:13.390623 14249 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.391085 14249 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:13.392670 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.396215 14226 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:13.048759 14394 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:16:13.046169 14392 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:16:13.047600 14390 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:16:13.043541 14395 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:16:13.037122 14396 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:13.043957 14383 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.402695 14237 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.204730 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:13.207419 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.414732 14239 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:13.421932 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.382843 14343 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.278384 13873 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:13.174369 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 343953us
W20260709 13:16:13.428916 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.428067 11298 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.430220 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:13.422394 14239 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.431564 14341 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:13.432277 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.041443 14299 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.445883 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.446871 11545 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:13.447541 11476 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:13.449497 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:13.450438 11542 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:13.432746 11298 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 4, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.444906 11470 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:13.439689 14239 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.460031 11473 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:13.465118 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:13.467650 14377 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:13.469532 14262 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.469668 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1248) took 2436 ms. Trace:
I20260709 13:16:13.470160 11470 rpcz_store.cc:276] 0709 13:16:11.033047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.620791 (+587744us) service_pool.cc:224] Handling call
0709 13:16:11.621120 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:13.376522 (+1755402us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5800812cd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da98750aa 00007f8da98f7447 00007f8da990ffe7 00007f8da99117de 00007f8da991bb49 00007f8da991c416 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4
0709 13:16:13.455419 (+ 78897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.469574 (+ 14155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860637440}
W20260709 13:16:13.471298 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 374) took 2328 ms (client timeout 1563 ms). Trace:
W20260709 13:16:13.471702 11396 rpcz_store.cc:269] 0709 13:16:11.143052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.143199 (+   147us) service_pool.cc:224] Handling call
0709 13:16:13.470858 (+2327659us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:13.471042 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610453248}
W20260709 13:16:13.473788 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 376) took 2329 ms (client timeout 2022 ms). Trace:
W20260709 13:16:13.474264 11402 rpcz_store.cc:269] 0709 13:16:11.143838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.574420 (+430582us) service_pool.cc:224] Handling call
0709 13:16:13.473340 (+1898920us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54003d4ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:13.473577 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247167360}
I20260709 13:16:13.482597 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.484166 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:13.484901 11549 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:13.484943 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.487314 12113 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:13.488693 14084 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.444324 13802 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:13.490118 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 611332us
I20260709 13:16:13.362568 11393 rpcz_store.cc:276] 0709 13:16:11.136034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.139471 (+  3437us) service_pool.cc:224] Handling call
0709 13:16:11.139870 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:11.140155 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.140195 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:11.167558 (+ 27363us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.753425 (+585867us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:11.753494 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.753586 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:13.190891 (+1437305us) raft_consensus.cc:1685] finished
0709 13:16:13.190973 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:13.191385 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":306887,"threads_started":1,"wal-append.queue_time_us":696983}
I20260709 13:16:13.367695 14386 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.506242 14362 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.508591 11298 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 4, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.510890 11298 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 5, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.490512 14361 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:13.515424 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.516788 14376 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.517868 14361 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:13.519083 14361 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:13.519565 14361 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:13.520829 14361 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.521315 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:13.524042 11542 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:13.525882 11549 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:13.533404 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.441612 11472 rpcz_store.cc:276] 0709 13:16:11.188623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.891918 (+703295us) service_pool.cc:224] Handling call
0709 13:16:12.098101 (+206183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:12.098306 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:12.386624 (+288318us) log.cc:844] Serialized 14 byte log entry
0709 13:16:12.836096 (+449472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:12.836147 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:12.836189 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:12.836262 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:12.836377 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":453707,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":41781,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":22392,"thread_start_us":686613,"threads_started":2,"wal-append.queue_time_us":489633}
I20260709 13:16:13.440164 11469 rpcz_store.cc:276] 0709 13:16:11.139371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.626563 (+487192us) service_pool.cc:224] Handling call
0709 13:16:11.652584 (+ 26021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:11.849367 (+196783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.849560 (+   193us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:11.886259 (+ 36699us) log.cc:844] Serialized 26 byte log entry
0709 13:16:12.321593 (+435334us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:12.321648 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:12.321738 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:12.553846 (+232108us) raft_consensus.cc:1685] finished
0709 13:16:12.553875 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:12.743982 (+190107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67420,"raft.queue_time_us":214324,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":2730,"thread_start_us":521333,"threads_started":2,"wal-append.queue_time_us":595805}
I20260709 13:16:13.547930 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1880) took 1017 ms. Trace:
I20260709 13:16:13.523485 14386 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.529958 14361 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:13.553184 14376 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.553906 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:13.554575 14403 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:13.559168 14393 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:16:13.560752 14303 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:13.561060 14393 raft_consensus.cc:2764] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:13.564163 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:13.565002 12116 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:13.544102 11545 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:13.578884 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.571139 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.548802 11476 rpcz_store.cc:276] 0709 13:16:12.530276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.449474 (+919198us) service_pool.cc:224] Handling call
0709 13:16:13.449999 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:13.450532 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:13.450945 (+   413us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.547462 (+ 96517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:13.547538 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:13.547597 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.547688 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:13.547841 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":11590,"thread_start_us":82606,"threads_started":1}
I20260709 13:16:13.582379 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:13.595299 13873 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.581923 11298 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 6, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 6 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:13.602669 14239 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:13.610749 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.635346 14239 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.617748 14286 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.599546 14376 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:13.618851 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 63891.9us
I20260709 13:16:13.619937 14386 raft_consensus.cc:2764] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:13.621866 14189 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.621245 11298 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 1 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.623337 11551 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:13.627530 14349 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:13.632268 14052 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.632931 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 552) took 2522 ms. Trace:
W20260709 13:16:13.646971 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 65220.2us
I20260709 13:16:13.654714 12112 rpcz_store.cc:276] 0709 13:16:11.110378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.114482 (+  4104us) service_pool.cc:224] Handling call
0709 13:16:11.114888 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:11.141771 (+ 26883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.632268 (+2490497us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54005cec54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:13.632678 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59871616}
I20260709 13:16:13.682878 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:13.683085 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 79500.5us
I20260709 13:16:13.645092 14249 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.709178 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.706089 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.700397 14239 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.713629 11470 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:13.661919 14189 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:13.718940 11551 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:13.724241 12113 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 468) took 1505 ms (client timeout 1901 ms). Trace:
I20260709 13:16:13.725189 11365 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:13.724557 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 862767us
W20260709 13:16:13.727105 12113 rpcz_store.cc:269] 0709 13:16:12.218829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:12.218956 (+   127us) service_pool.cc:224] Handling call
0709 13:16:13.715346 (+1496390us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540043d854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:13.724146 (+  8800us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2419083136}
I20260709 13:16:13.729707 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:13.730007 14371 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 13:16:13.730780 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 718754us
W20260709 13:16:13.731894 11514 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.059s (SENT)
I20260709 13:16:13.732416 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:13.732985 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.733839 11544 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 2.
I20260709 13:16:13.739357 12119 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:13.744287 12116 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:13.745183 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:13.746047 12112 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:16:13.747922 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.730804 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.771649 11471 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:13.776535 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.777505 11470 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:13.733035 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.779111 11468 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:13.779310 14330 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:13.780732 14330 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:13.781085 14052 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.786130 14419 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:13.711525 14376 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.724941 13802 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:16:13.735427 11469 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:13.791234 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 89450.2us
I20260709 13:16:13.793509 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.794454 11477 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:13.822611 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 90945.5us
W20260709 13:16:13.824127 11514 leader_election.cc:336] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.131s (SENT)
I20260709 13:16:13.824594 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:13.826915 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:13.781111 14330 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:13.803435 14299 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:13.796564 14352 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.842274 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:13.843734 14264 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.843834 14262 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.849125 14249 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.850531 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:13.855896 14278 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 2.117s	user 0.010s	sys 0.010s
W20260709 13:16:13.856009 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.871613 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.872623 11393 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:13.874095 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:13.875865 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:13.856616 14335 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:13.877192 14335 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.856199 14264 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.857661 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 467) took 1639 ms. Trace:
I20260709 13:16:13.880326 12115 rpcz_store.cc:276] 0709 13:16:12.218489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:12.310435 (+ 91946us) service_pool.cc:224] Handling call
0709 13:16:12.504418 (+193983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.240844 (+736426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.857179 (+616335us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54005ff0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:13.857477 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768093056}
I20260709 13:16:13.858078 11470 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:13.859822 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:13.881544 11470 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:13.882431 11470 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:13.883387 11470 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:13.861276 14418 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
W20260709 13:16:13.861370 14427 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.093s	user 0.000s	sys 0.004s
I20260709 13:16:13.882834 14264 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.861814 11471 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:13.889351 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.890331 11398 raft_consensus.cc:2410] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:13.891366 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.891707 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:13.866308 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.827246 13873 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.839753 14239 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:13.895730 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:13.941784 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.981139 11471 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:13.919206 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.850132 14299 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:13.904115 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:13.967355 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 162047us
I20260709 13:16:13.994251 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.995733 14348 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:13.966224 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:13.867003 14052 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.885877 14335 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:14.015153 14356 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.003s
I20260709 13:16:14.036155 14356 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.038141 14358 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.038448 14356 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.018450 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.046241 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.019445 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.047698 11402 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.048709 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.049510 11402 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:14.020370 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.050493 11398 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.002396 14343 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:14.014492 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.021416 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.023626 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.024245 14146 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:14.022554 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.035360 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:13.991366 14299 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:13.918547 14286 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:13.967046 14237 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.956321 14376 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:13.942967 14396 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:14.032863 14262 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.069870 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.071568 11545 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:14.072294 11395 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:14.073376 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:14.074120 11471 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:14.000041 14372 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.185s
I20260709 13:16:14.092085 11393 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:14.097080 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.099484 11394 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.011062 13873 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.110877 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:14.113330 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.087036 14343 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:14.074589 14052 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.123037 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:14.111236 14422 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:16:14.125641 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.127924 11471 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.074782 14237 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.135916 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.136920 11548 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:14.139233 13873 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.139460 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.151232 11542 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.057996 14330 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.156802 14367 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:14.157384 14367 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:14.158315 14367 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.160084 14393 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:14.160535 14393 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:14.161562 14393 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.162604 14393 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:14.163877 14343 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:14.156911 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.157032 14330 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:14.176575 11514 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.113s (SENT)
I20260709 13:16:14.177786 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 466) took 2017 ms. Trace:
I20260709 13:16:14.178380 12119 rpcz_store.cc:276] 0709 13:16:12.160322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:12.307727 (+147405us) service_pool.cc:224] Handling call
0709 13:16:12.332757 (+ 25030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.739886 (+1407129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.177473 (+437587us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400720454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:14.177725 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3228128640}
W20260709 13:16:14.177886 11514 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.171s (SENT)
I20260709 13:16:14.179644 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.180420 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.181500 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.182032 11548 raft_consensus.cc:2376] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8. Re-sending same reply.
W20260709 13:16:14.180346 11514 leader_election.cc:336] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.199s (SENT)
W20260709 13:16:14.192727 11473 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.000s
I20260709 13:16:14.198633 14367 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.201550 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.202224 11548 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:14.204133 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.205005 11547 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.206624 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.207309 11549 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:14.208539 12085 connection.cc:256] RPC call timeout handler was delayed by -1.8240183870000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:16:14.208985 12085 connection.cc:256] RPC call timeout handler was delayed by -1.5657183870000608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 13:16:14.209270 12085 connection.cc:256] RPC call timeout handler was delayed by -1.4109183870000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
W20260709 13:16:14.209558 12085 connection.cc:256] RPC call timeout handler was delayed by -1.4109183870000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 13:16:14.195923 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.196480 11396 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.196952 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.213011 11394 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:14.224324 11473 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:16:14.224967 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.225510 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.229105 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:14.224673 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.247362 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 425) took 1416 ms. Trace:
I20260709 13:16:14.212165 14146 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.248584 11469 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:14.247905 11473 rpcz_store.cc:276] 0709 13:16:12.831180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.203935 (+372755us) service_pool.cc:224] Handling call
0709 13:16:14.247291 (+1043356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.248256 14435 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:14.264195 14431 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:16:14.265072 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 84784.3us
W20260709 13:16:14.265803 11514 leader_election.cc:336] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.287s (SENT)
I20260709 13:16:14.266173 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:14.267036 14435 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:16:14.267267 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0894288440001674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2596s for a response
W20260709 13:16:14.267709 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0534288440001092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2636s for a response
I20260709 13:16:14.269726 14299 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.274999 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.275285 11399 raft_consensus.cc:2368] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
W20260709 13:16:14.273496 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.291342 14146 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.282617 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.298580 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.300835 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.301707 11542 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:14.316443 14146 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:14.318037 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.319201 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66976.9us
I20260709 13:16:14.322523 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.302925 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.323951 11542 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.324287 14052 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:14.325171 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.326041 11542 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:16:14.327214 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:14.329164 14257 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.333139 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.334427 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.222846 14372 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:16:14.355470 11471 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:14.347991 11547 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:14.333837 11400 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:14.351836 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.352521 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:14.373530 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.376128 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.381048 11471 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:14.382239 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.382941 11542 raft_consensus.cc:2376] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7. Re-sending same reply.
I20260709 13:16:14.384013 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.384603 11549 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:14.385360 11400 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:14.383592 12115 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:14.387189 12119 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:14.390901 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.393416 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.394318 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.397661 11548 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.393416 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.405244 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.405616 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.406212 11474 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:14.406708 11549 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:14.408399 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.408541 14299 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:14.409821 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 240759us
I20260709 13:16:14.409729 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.427249 11549 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:14.431429 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:14.430671 11514 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.596s (SENT)
I20260709 13:16:14.443598 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:14.446352 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:14.434644 14393 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:14.448967 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 353763us
I20260709 13:16:14.445607 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:14.451071 11395 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:14.452157 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.429884 14257 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:14.458653 11298 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.009s	sys 0.030s
I20260709 13:16:14.463349 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:14.464639 11298 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.010s	sys 0.030s
W20260709 13:16:14.462385 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.466684 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.466253 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.468080 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:14.468326 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.477545 11470 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:14.459800 14421 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.689s	user 0.000s	sys 0.006s
I20260709 13:16:14.479672 11471 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:16:14.479499 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:14.480425 11471 pending_rounds.cc:85] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 4
I20260709 13:16:14.480825 11471 pending_rounds.cc:107] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:16:14.481295 11471 raft_consensus.cc:2889] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:14.481776 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51143.9us
W20260709 13:16:14.483232 11514 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.636s (SENT)
W20260709 13:16:14.484226 11514 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.698s (SENT)
I20260709 13:16:14.458745 14257 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:14.463505 14146 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.492334 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.494421 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.512542 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.513370 11549 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:14.509347 11469 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:14.516330 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:14.523064 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:14.456614 14393 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.527276 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.527830 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.528784 11549 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:16:14.530437 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 59860.3us
I20260709 13:16:14.527941 12115 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.507897 14257 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.533741 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:14.536659 11395 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:14.538908 11477 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:14.539225 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.545434 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1288) took 3357 ms. Trace:
I20260709 13:16:14.527196 14372 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:14.554270 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.559712 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.559978 11397 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:14.563778 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.564872 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.567457 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.568817 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:14.572746 12115 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:14.574286 11469 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:14.545853 11401 rpcz_store.cc:276] 0709 13:16:11.136307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.152866 (+ 16559us) service_pool.cc:224] Handling call
0709 13:16:11.164673 (+ 11807us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:11.865643 (+700970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.865685 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:13.485796 (+1620111us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b180035ef60. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a
0709 13:16:13.486840 (+  1044us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:13.572126 (+ 85286us) log.cc:844] Serialized 52 byte log entry
0709 13:16:13.695602 (+123476us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:13.695651 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.044336 (+348685us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b5400126bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:14.044390 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.272741 (+228351us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:14.493833 (+221092us) raft_consensus.cc:1685] finished
0709 13:16:14.493854 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.493992 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47133,"raft.queue_time_us":18068,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":13091,"spinlock_wait_cycles":1755867136,"thread_start_us":105722,"threads_started":1,"wal-append.queue_time_us":180779}
I20260709 13:16:14.547210 14435 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:14.577502 11395 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:14.580915 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.552561 14372 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:14.577821 12113 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:14.582604 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:14.583333 11476 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:14.585590 12116 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:14.587415 14353 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:14.587144 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.598275 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:14.596413 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 112205us
I20260709 13:16:14.600858 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.585425 14372 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.606149 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.606623 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:14.606840 12118 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:14.607282 12114 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:14.608918 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.609341 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.611063 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:14.611653 11472 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:14.615820 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:14.616451 11397 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.595333 14360 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:14.620221 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.584281 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.626410 11397 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:14.628521 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.571167 14356 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:16:14.635030 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:14.643253 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 63534.6us
I20260709 13:16:14.630599 11367 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.646324 11470 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:14.600911 14335 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:14.655521 11400 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:14.655521 11393 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:14.656703 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.657783 11469 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:14.619274 14360 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.635337 14297 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:14.675964 11472 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:14.687449 11401 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:16:14.695470 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.662936 14383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.015s	sys 0.048s
I20260709 13:16:14.669600 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.669729 11476 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:14.672166 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:14.666888 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45223.4us
I20260709 13:16:14.710947 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.605265 14435 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.671468 14343 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:14.686301 14360 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.686645 14297 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.672621 14335 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:14.729321 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:14.718894 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 49412.7us
I20260709 13:16:14.667470 13802 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:14.705739 14383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.017s	sys 0.048s
I20260709 13:16:14.732445 11516 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.735652 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.696416 14356 consensus_queue.cc:1243] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:14.722396 14239 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:14.722839 14343 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.730541 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.729844 14297 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.739950 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:14.734102 13873 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:14.745329 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 316899us
I20260709 13:16:14.745711 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.731416 13802 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:14.740100 14343 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:14.733174 14189 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:14.738741 14239 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:14.742235 14367 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:14.759974 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.741990 14393 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.747757 14052 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:14.747190 13873 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.751119 13802 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.764559 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:14.768648 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46681.1us
W20260709 13:16:14.771637 14440 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:14.772815 14435 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.760231 14239 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:14.777738 11396 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
I20260709 13:16:14.779318 11396 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.762553 14367 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:14.768826 13873 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.768653 14348 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:14.786684 14367 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:14.807376 14439 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:16:14.812620 14441 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.161s	user 0.001s	sys 0.001s
I20260709 13:16:14.814646 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.815280 11551 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.816462 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.817741 14303 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:14.818176 14303 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:14.794860 14335 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.819092 14303 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.820143 14303 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.824049 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.828187 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.778558 14257 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.839308 11401 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:14.839721 11514 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 4.590s (SENT)
I20260709 13:16:14.845495 13873 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.847922 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:14.848824 14457 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:14.849423 13818 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:14.849399 12114 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:14.860924 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:14.850158 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:14.865944 11400 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:14.798746 14348 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:14.867631 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.867384 14348 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.860024 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:14.872424 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.851069 14297 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.874882 14464 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.876291 14297 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.893828 14464 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:14.894769 14464 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.878113 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.896823 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:14.897630 14335 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:14.898159 14335 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.898538 14335 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.899914 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:14.900386 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.901134 11545 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:14.903189 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53112.3us
I20260709 13:16:14.903940 14264 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:14.904685 14264 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:14.905932 14264 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.909114 11545 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:14.873618 14239 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.891281 14338 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:14.910436 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.911053 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.911207 14338 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:14.912115 14338 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.912312 14257 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.912582 14338 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.913800 14286 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.866099 14366 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:14.901129 14464 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:14.915216 14366 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.915647 14366 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:14.917725 14429 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.810s	user 0.002s	sys 0.000s
I20260709 13:16:14.917531 14372 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:14.920941 14423 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.150s	user 0.002s	sys 0.004s
I20260709 13:16:14.845495 14343 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:14.862380 14457 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.923735 14457 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:14.925590 14343 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.926890 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1872) took 1445 ms. Trace:
W20260709 13:16:14.927826 14428 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.159s	user 0.000s	sys 0.006s
I20260709 13:16:14.929222 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1871) took 1448 ms. Trace:
W20260709 13:16:14.929335 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.930537 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.929932 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.930871 13873 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:14.932053 14330 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:14.932224 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 103141us
I20260709 13:16:14.929633 11543 rpcz_store.cc:276] 0709 13:16:13.480443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.483143 (+  2700us) service_pool.cc:224] Handling call
0709 13:16:13.483669 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.499152 (+ 15483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.499192 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.545274 (+ 46082us) log.cc:844] Serialized 26 byte log entry
0709 13:16:13.719087 (+173813us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:13.719134 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.719201 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.927233 (+1208032us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:14.928809 (+  1576us) raft_consensus.cc:1685] finished
0709 13:16:14.928835 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.929061 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28483,"raft.queue_time_us":17974,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":189,"spinlock_wait_cycles":119550592,"thread_start_us":61085,"threads_started":1,"wal-append.queue_time_us":181216}
W20260709 13:16:14.934921 11475 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 1873) took 2406 ms (client timeout 2238 ms). Trace:
W20260709 13:16:14.935376 11475 rpcz_store.cc:269] 0709 13:16:12.528271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:12.542474 (+ 14203us) service_pool.cc:224] Handling call
0709 13:16:14.934568 (+2392094us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:14.934720 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4608667904}
I20260709 13:16:14.935951 14237 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:14.938062 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.938357 11393 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:14.939386 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.939666 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.940354 11393 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.941196 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.941267 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.942548 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.892133 13818 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:14.957902 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.962908 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.963783 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.963649 13818 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.945864 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:14.958748 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.965405 11393 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:14.958496 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:14.991870 11550 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:14.964578 11542 raft_consensus.cc:2376] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5. Re-sending same reply.
I20260709 13:16:14.965018 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.992594 11548 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:14.955883 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:14.965050 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:14.974485 14433 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.773s	user 0.003s	sys 0.000s
I20260709 13:16:14.934979 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.911100 11545 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:16:14.971328 14465 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.093s	user 0.000s	sys 0.005s
W20260709 13:16:14.943485 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.003917 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 48106.5us
I20260709 13:16:14.956380 14360 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.969503 14367 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.968019 13802 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:14.965147 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.942646 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:14.878917 11551 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:14.990332 11402 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:15.026546 11397 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:15.023694 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:14.997983 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:14.927434 11546 rpcz_store.cc:276] 0709 13:16:13.480963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.483223 (+  2260us) service_pool.cc:224] Handling call
0709 13:16:13.483777 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.500731 (+ 16954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.500766 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.551970 (+ 51204us) log.cc:844] Serialized 26 byte log entry
0709 13:16:13.712151 (+160181us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:13.712195 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.712273 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.920219 (+1207946us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:14.926375 (+  6156us) raft_consensus.cc:1685] finished
0709 13:16:14.926403 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.926664 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9072,"raft.queue_time_us":17996,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11491,"spinlock_wait_cycles":98270336,"thread_start_us":96742,"threads_started":1,"wal-append.queue_time_us":184010}
I20260709 13:16:15.028021 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:15.028455 14420 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.238s	user 0.000s	sys 0.005s
I20260709 13:16:15.028729 11543 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.029138 14239 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:14.995546 14262 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.008252 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:15.008167 14415 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.273s	user 0.005s	sys 0.001s
I20260709 13:16:15.034845 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:14.998965 14383 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:15.037973 14383 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.038697 14146 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:15.041167 14366 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.041975 11402 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:15.018731 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:15.013536 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.017853 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.020553 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.044592 11516 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:15.046052 14383 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.045382 14396 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:16:15.054436 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.054801 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:15.055507 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 530854us
I20260709 13:16:15.057778 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:15.058630 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1292) took 1188 ms. Trace:
I20260709 13:16:15.059126 11394 raft_consensus.cc:2368] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:15.059392 11398 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:15.049441 11544 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:15.052237 11393 raft_consensus.cc:2368] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:16:15.059002 11396 rpcz_store.cc:276] 0709 13:16:13.870548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.875534 (+  4986us) service_pool.cc:224] Handling call
0709 13:16:15.058486 (+1182952us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54001270d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:15.058579 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735580416}
I20260709 13:16:15.011264 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.028131 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.005511 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.063352 11542 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:16:15.028048 14432 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.780s	user 0.005s	sys 0.000s
W20260709 13:16:15.067767 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 56632.9us
I20260709 13:16:15.069372 14299 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:15.069906 11402 raft_consensus.cc:2368] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:15.070086 14299 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.070973 14299 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.072113 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.072793 11549 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:15.073593 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:15.074131 11549 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:15.074803 14352 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:15.075235 14352 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:15.069463 14382 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:15.075624 14352 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:15.079615 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.075412 14338 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.075657 14382 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:15.088972 14372 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.090051 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:15.090852 11546 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:15.091665 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.092388 14338 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.084144 11548 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:15.088461 14348 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:15.089694 14382 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:15.082940 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 298461us
I20260709 13:16:15.046270 14239 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.098266 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:15.050541 13802 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:15.122733 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.122975 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 223116us
I20260709 13:16:15.128603 11551 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:15.134045 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:14.946545 14464 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.096217 14382 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.042958 14335 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:15.147637 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63849.1us
I20260709 13:16:15.095921 14348 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.154820 11550 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:15.155856 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:15.164443 11399 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.157074 11395 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:15.094947 14360 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.159508 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.109609 14366 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.167260 11547 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.166006 11542 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:15.143122 14338 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:15.181499 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.268674 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.227090 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:15.233817 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.260308 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.269896 12116 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:16:15.269933 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.166708 14348 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:15.262550 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 82247.5us
I20260709 13:16:15.265069 11545 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 4.
I20260709 13:16:15.131901 14297 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:15.175984 13802 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:15.272374 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:15.272845 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 92892.3us
I20260709 13:16:15.270036 12117 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:15.099638 14146 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:16:15.274135 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 116867us
I20260709 13:16:15.276638 11542 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.270568 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.277875 12115 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:15.281457 12118 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:15.277942 12112 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:15.288941 11547 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:15.276484 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.298756 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.315533 11543 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:15.316035 11542 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:15.318269 11471 pending_rounds.cc:85] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 9
I20260709 13:16:15.318624 11471 pending_rounds.cc:107] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:16:15.318881 11471 raft_consensus.cc:2889] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:15.305701 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.323668 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.325037 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.326623 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.327571 14435 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:15.327797 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.328143 14435 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.329468 11548 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:15.330235 14435 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:15.330309 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.331946 14303 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:15.332374 14303 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:15.332742 14303 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:15.336196 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 59814.4us
I20260709 13:16:15.336536 14376 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.337007 11366 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.338753 11365 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.339622 14257 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:15.339969 14257 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.340291 14257 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:15.340512 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.341224 14189 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.341619 11365 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:15.342311 14189 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:15.339039 13818 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:15.338528 14455 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:15.343904 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.354163 11545 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.345458 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:15.338616 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:15.351554 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.268731 14338 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:15.273236 13802 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:15.354300 11547 raft_consensus.cc:1217] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Deduplicated request from leader. Original: 3.2->[4.3-6.4]   Dedup: 6.4->[]
I20260709 13:16:15.181227 14461 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
I20260709 13:16:15.359057 11547 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:16:15.361518 11543 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:15.365075 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.365499 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.371307 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:15.348775 12114 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:15.317444 14330 raft_consensus.cc:2764] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:16:15.353864 14455 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.365409 14338 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.393677 11547 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:15.359031 14446 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:15.290647 14297 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.366250 13802 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.342902 14376 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
W20260709 13:16:15.402149 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 277000us
W20260709 13:16:15.412212 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.381263 14393 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:15.381273 14383 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:15.358258 14237 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:15.413748 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 83487.8us
I20260709 13:16:15.398900 14455 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.429171 14464 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.435082 14393 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.482257 14393 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.482690 14303 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.402112 13873 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:15.389307 14343 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:15.411129 14457 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.485212 13873 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:15.485827 14460 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.226s	user 0.002s	sys 0.000s
I20260709 13:16:15.486333 13873 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.487223 14457 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.457409 11399 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:15.460476 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.489226 11544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1779) took 1693 ms (client timeout 2145 ms). Trace:
W20260709 13:16:15.489640 11544 rpcz_store.cc:269] 0709 13:16:13.795677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.739112 (+943435us) service_pool.cc:224] Handling call
0709 13:16:15.489015 (+749903us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54000b2054. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:15.489146 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":692616448}
W20260709 13:16:15.492413 14471 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:16:15.464048 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 137499us
I20260709 13:16:15.410154 14367 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:15.464080 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.500181 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.500480 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.501163 14303 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.501551 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.501799 14303 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:15.498528 14382 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:15.502938 14343 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:15.503065 14372 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:15.503484 14343 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.503644 14372 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.504093 14372 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.504431 14343 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.477083 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:15.477227 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.477996 14052 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:15.477811 14237 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.479259 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:15.480590 14470 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:16:15.481381 14467 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:16:15.481458 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.530879 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.514532 11545 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
W20260709 13:16:15.509341 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 136357us
I20260709 13:16:15.532732 11401 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:15.529167 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.535657 11542 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.463155 14383 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:15.558014 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.560549 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.464712 14446 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:15.563570 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 63465.1us
I20260709 13:16:15.564337 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:15.562525 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 190502us
I20260709 13:16:15.507652 14303 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:15.565398 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.523684 14052 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.565945 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:15.568589 11545 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.570497 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 559) took 1104 ms. Trace:
W20260709 13:16:15.565090 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 215109us
I20260709 13:16:15.567668 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.507279 14264 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:16:15.564872 14383 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.574963 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.578827 11542 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:15.571491 12122 rpcz_store.cc:276] 0709 13:16:14.465859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.470791 (+  4932us) service_pool.cc:224] Handling call
0709 13:16:14.511228 (+ 40437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.524034 (+ 12806us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:14.714084 (+190050us) log.cc:844] Serialized 14 byte log entry
0709 13:16:15.324218 (+610134us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18002a2440. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:16:15.570040 (+245822us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:15.570105 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:15.570161 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.570240 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.570431 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49357,"prepare.queue_time_us":213255,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1665,"spinlock_wait_cycles":3711872,"thread_start_us":376127,"threads_started":2}
I20260709 13:16:15.578739 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.565071 14297 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.560734 14335 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.590849 12117 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:16:15.597585 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.591774 11545 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:15.593729 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 251022us
I20260709 13:16:15.571486 14286 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:15.577803 14464 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:15.614145 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.615371 11473 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:15.614758 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.619449 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.623095 14475 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.625337 12122 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:15.626719 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.627048 14335 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:15.627367 12123 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.627774 14482 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:15.628149 14482 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:15.628490 14335 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:15.628923 14482 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.630085 14482 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.627317 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.633073 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.633422 14372 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.634604 11398 raft_consensus.cc:2368] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:15.634749 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.635107 14303 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.638397 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.639194 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.640978 12118 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:15.643771 11401 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:16:15.611750 14297 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:15.642526 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 67146.1us
I20260709 13:16:15.649308 12123 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:15.627454 14481 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.621048 14464 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:15.657117 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.654661 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:15.674096 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:15.679695 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:15.676651 12118 raft_consensus.cc:2376] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9. Re-sending same reply.
I20260709 13:16:15.648733 14189 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.642001 14348 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:15.718209 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 98911.8us
I20260709 13:16:15.720837 14480 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.721628 14476 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.722496 14476 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:15.722970 14476 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:15.723366 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1908) took 1387 ms. Trace:
I20260709 13:16:15.724501 14372 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.725639 14338 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.728751 14348 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.730724 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.715493 14257 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.732648 14382 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:15.733898 14382 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.734417 14382 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.734800 14257 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.736349 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.744725 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.723796 11551 rpcz_store.cc:276] 0709 13:16:14.336320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.150721 (+814401us) service_pool.cc:224] Handling call
0709 13:16:15.225465 (+ 74744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:15.292384 (+ 66919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:15.292415 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:15.376100 (+ 83685us) log.cc:844] Serialized 26 byte log entry
0709 13:16:15.559085 (+182985us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:15.571081 (+ 11996us) log.cc:844] Serialized 14 byte log entry
0709 13:16:15.571660 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.571733 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.723085 (+151352us) raft_consensus.cc:1685] finished
0709 13:16:15.723111 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.723292 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14565,"prepare.queue_time_us":12126,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":39171,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":4356,"thread_start_us":147041,"threads_started":1,"wal-append.queue_time_us":245561}
I20260709 13:16:15.752720 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 388) took 1330 ms. Trace:
I20260709 13:16:15.736011 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.757673 11400 rpcz_store.cc:276] 0709 13:16:14.422239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.665922 (+243683us) service_pool.cc:224] Handling call
0709 13:16:14.666316 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.866373 (+200057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.752469 (+886096us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540019a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:15.752655 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458631424}
W20260709 13:16:15.775701 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 121105us
I20260709 13:16:15.674089 13802 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.802317 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.803184 11401 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
W20260709 13:16:15.743454 14472 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:16:15.804766 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.752898 14352 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.003s
I20260709 13:16:15.750865 14492 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:15.807667 11398 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:16:15.757212 14264 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:16:15.800384 14423 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:16:15.739583 14338 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:15.740026 14382 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.734932 14393 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.814858 11548 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:15.743929 14237 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.745098 14189 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.824564 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.810539 14475 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:15.825987 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.826885 14496 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:15.827343 14491 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:15.827806 11442 connection.cc:256] RPC call timeout handler was delayed by -4.9408955550002247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 13:16:15.828121 14393 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.747342 14464 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.829517 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.890084 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:15.891081 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.891263 12119 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.676549 14481 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.891716 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:15.879968 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55500us
W20260709 13:16:15.850739 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.881954 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.816497 14492 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:15.896502 14498 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:16:15.897043 14492 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.897567 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.898649 14492 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.888557 11474 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:15.903527 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:15.903849 14478 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:16:15.904197 11471 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:15.905370 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:15.905980 11471 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:15.892848 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.878693 14264 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.904238 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.818256 14352 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.908051 13802 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.907819 14264 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.899489 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.899780 12114 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:15.909714 11546 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:15.895489 14348 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:16:15.887672 11442 connection.cc:256] RPC call timeout handler was delayed by -4.9372955550002189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:16:15.910369 11442 connection.cc:256] RPC call timeout handler was delayed by -4.915695555000184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:16:15.910413 14352 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:15.910567 14348 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.823304 14052 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.911679 14348 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.912339 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.912787 14052 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.913437 14052 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.901047 11401 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:15.723919 14476 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.901774 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:15.910856 11442 connection.cc:256] RPC call timeout handler was delayed by -4.8904955550001432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 13:16:15.917171 11442 connection.cc:256] RPC call timeout handler was delayed by -4.8733955550001156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:16:15.917512 11442 connection.cc:256] RPC call timeout handler was delayed by -4.8481955550000748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 13:16:15.917827 11442 connection.cc:256] RPC call timeout handler was delayed by -4.8292955550000443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 13:16:15.918160 11442 connection.cc:256] RPC call timeout handler was delayed by -4.6987955550000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 13:16:15.918501 11442 connection.cc:256] RPC call timeout handler was delayed by -4.6249955550001687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 13:16:15.889192 14286 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:15.920440 14286 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:15.908171 11474 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.921442 14286 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.923668 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:15.924326 11398 raft_consensus.cc:2376] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10. Re-sending same reply.
I20260709 13:16:15.925390 11470 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.926348 14481 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:15.926689 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:15.927590 14457 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:16:15.918896 11442 connection.cc:256] RPC call timeout handler was delayed by -4.5970955550001236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 13:16:15.930764 11442 connection.cc:256] RPC call timeout handler was delayed by -4.4836955550001676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 13:16:15.931165 11442 connection.cc:256] RPC call timeout handler was delayed by -4.3936955550000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
I20260709 13:16:15.893246 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.892257 14496 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.932183 11469 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.932313 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.933151 11468 raft_consensus.cc:2368] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9.
I20260709 13:16:15.933658 14496 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:15.935276 11442 connection.cc:256] RPC call timeout handler was delayed by -4.3243955550001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 13:16:15.907917 14455 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:15.936144 11442 connection.cc:256] RPC call timeout handler was delayed by -4.2964955550000923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
W20260709 13:16:15.936960 11442 connection.cc:256] RPC call timeout handler was delayed by -3.9688955550002447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.258s for a response
W20260709 13:16:15.939553 11442 connection.cc:256] RPC call timeout handler was delayed by -3.9463955550002083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2605s for a response
W20260709 13:16:15.940332 11442 connection.cc:256] RPC call timeout handler was delayed by -3.7879955550001796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2781s for a response
W20260709 13:16:15.940754 11442 connection.cc:256] RPC call timeout handler was delayed by -3.6088955550001174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.298s for a response
W20260709 13:16:15.941174 11442 connection.cc:256] RPC call timeout handler was delayed by -3.5539955550000286s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30410000000000004s for a response
I20260709 13:16:15.941844 11470 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
W20260709 13:16:15.941943 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50306.6us
I20260709 13:16:15.935317 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.943405 12113 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.943840 11473 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.906932 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.944866 11471 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:15.946214 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:15.947870 11393 raft_consensus.cc:2368] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:15.958856 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1785) took 1031 ms. Trace:
I20260709 13:16:15.959125 14257 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:15.959319 11546 rpcz_store.cc:276] 0709 13:16:14.927296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.091436 (+164140us) service_pool.cc:224] Handling call
0709 13:16:15.958654 (+867218us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b540041d7d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:15.958782 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1879697152}
I20260709 13:16:15.960228 14353 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:15.960407 14257 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:15.960670 14353 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:15.960969 14257 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:15.961705 14353 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.961966 14257 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.965287 12113 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:15.960601 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.967833 11473 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:15.974509 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:15.975867 11476 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:16:15.976282 11476 pending_rounds.cc:85] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:16:15.976533 11476 pending_rounds.cc:107] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:16:15.976773 11476 raft_consensus.cc:2889] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:15.972601 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:15.978670 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:15.991161 11516 leader_election.cc:400] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:16:15.991632 11516 leader_election.cc:403] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:15.992242 14457 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:15.889179 14382 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:15.966073 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:15.997613 12085 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:15.962823 14353 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:16.005832 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:16.010617 11442 leader_election.cc:400] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:16:16.011300 11442 leader_election.cc:403] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:16.008760 14466 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:16:16.013037 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:16.008751 14382 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.016510 14382 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:16.004959 14475 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:16:16.017993 14475 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:16.018384 14475 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:16.019930 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:16.020520 11470 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:16.021520 14383 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:15.963718 14303 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:16.023108 14257 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:16.027542 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:16.026839 11470 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:16.028216 11476 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:16.032224 11469 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:16.034687 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 68676us
W20260709 13:16:16.075555 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.089757 14500 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:16.094444 12119 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:16.095116 11474 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:16.100347 12114 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:16.105108 11471 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:16.107689 14383 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:16.097138 14303 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:16.112743 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:16.113687 14491 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:16.114249 14491 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:16.113747 14303 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:15.968981 14338 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:16:16.115278 14491 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.118530 11393 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:16.119844 11476 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:16.128440 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.251785 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.275063 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 183032us
I20260709 13:16:16.278735 11365 leader_election.cc:400] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9.
I20260709 13:16:16.279201 11365 leader_election.cc:403] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:16.105726 14481 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:16.299532 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 289119us
I20260709 13:16:16.096072 14382 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5.
I20260709 13:16:16.300585 14481 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.275558 14383 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.314373 11470 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:16.298792 14299 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.278932 14338 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:16:16.320211 14375 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.338755 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.355346 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:16.356889 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.358196 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.323338 14383 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:16.378146 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 569692us
I20260709 13:16:16.379357 14496 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:16.386125 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.399034 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 435517us
I20260709 13:16:16.403648 11516 leader_election.cc:400] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:16.405113 11474 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:16.338748 14299 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.380630 14338 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:16.404129 11516 leader_election.cc:403] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:16:16.411105 14499 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:16.411774 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.412146 14367 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:16.413017 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.420156 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:16.381790 14382 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:16.427927 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:16.428483 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:16.429335 11545 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:16.385604 14487 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:16.428733 11542 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:16.436525 11516 leader_election.cc:400] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:16.437001 11516 leader_election.cc:403] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:16.385339 14492 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:16.404052 14239 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:16:16.452368 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:16.454661 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:16.458725 11550 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:16.459838 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.460260 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 184338us
I20260709 13:16:16.391294 13802 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 13:16:16.468544 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.470727 11542 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:16.473505 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.464883 11365 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:16.476099 11365 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:16.417994 14338 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:16.477950 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44913.6us
I20260709 13:16:16.479367 11543 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:16.424446 14481 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:16.491833 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:16.492446 11545 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:16.497262 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:16.440443 14353 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:16.435153 14487 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:16.500586 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:16.451825 14393 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:16.477114 14499 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.493104 13873 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:16.501037 14497 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
I20260709 13:16:16.441794 14367 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.441290 14237 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:16:16.498364 14502 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:16:16.523036 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 115946us
I20260709 13:16:16.524308 11550 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:16.524827 11545 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:16.525780 11546 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:16.513193 14487 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:16.522122 14505 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:16:16.512842 14393 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:16.503741 14435 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:16.515545 14239 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:16.527727 14257 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:16.535840 13873 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:16.534267 14237 consensus_queue.cc:1243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:16.544224 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:16.557917 14475 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.559442 14375 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9.
I20260709 13:16:16.536408 14393 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:16.536980 14457 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:16.545884 14508 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:16:16.546260 14383 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:16.522825 14499 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.551563 14299 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:16.551887 14338 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.553042 14367 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:16.545167 13873 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.606886 11400 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:16.620069 11395 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:16.601079 14382 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:16.595937 14257 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:16.627993 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 159241us
W20260709 13:16:16.629587 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92144.6us
I20260709 13:16:16.606390 14457 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:16.631385 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 141645us
I20260709 13:16:16.634160 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.635159 14520 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:16:16.635903 14338 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:16.636464 14286 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:16.636885 14286 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:16.637478 14257 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.637727 14286 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.644366 14499 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:16.647727 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:16.648603 11545 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:16.650187 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:16.649874 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 137264us
I20260709 13:16:16.667397 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:16.668174 11545 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:16.672175 11546 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 13:16:16.672659 11546 pending_rounds.cc:85] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 8
I20260709 13:16:16.672930 11546 pending_rounds.cc:107] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 13:16:16.673375 11546 raft_consensus.cc:2889] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:16.675922 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:16.649016 14286 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.674882 14435 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.655966 14382 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:16.687508 11477 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:16.688309 14531 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:16:16.688095 14435 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.690519 14435 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:16.691780 11469 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:16.693401 11476 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:16.694718 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:16.695434 11472 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.696916 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.698395 11442 leader_election.cc:400] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:16:16.698684 11442 leader_election.cc:403] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:16:16.668985 14517 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:16:16.700795 14499 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:16.701665 14499 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6.
I20260709 13:16:16.703405 14457 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.704473 14455 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:16.708213 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.713761 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:16.716357 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:16.719864 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 87101.1us
I20260709 13:16:16.718743 11472 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:16.727202 11469 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:16.740025 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:16.742269 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.743112 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:16.743793 12113 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:16.720410 14464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.020s	sys 0.008s
I20260709 13:16:16.748739 11474 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:16.753050 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:16.729986 14383 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:16.748543 14446 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:16:16.729243 14461 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:16:16.752066 14464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.021s	sys 0.008s
I20260709 13:16:16.758599 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:16.757822 11472 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:16.793782 11477 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:16:16.794150 11477 pending_rounds.cc:85] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 3
I20260709 13:16:16.794395 11477 pending_rounds.cc:107] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:16.794615 11477 raft_consensus.cc:2889] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:16.796182 12113 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:16.764333 14487 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.801812 11474 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:16:16.803647 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.757184 14499 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:16.812135 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.763312 14383 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:16.781234 14237 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.811591 11477 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:16.816797 11477 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:16.820086 14237 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:16.822487 14237 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:16.826016 14534 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:16:16.828195 14476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.035s	sys 0.025s
I20260709 13:16:16.811350 14482 raft_consensus.cc:2764] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:16:16.809777 14455 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:16.764691 14393 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:16.831076 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72475.9us
I20260709 13:16:16.835500 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:16.837842 11471 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:16.830569 14476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.035s	sys 0.025s
I20260709 13:16:16.843924 14383 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.846232 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 1931) took 1317 ms. Trace:
I20260709 13:16:16.846664 11548 rpcz_store.cc:276] 0709 13:16:15.529164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.529372 (+   208us) service_pool.cc:224] Handling call
0709 13:16:16.845971 (+1316599us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540041c3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:16.846136 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582443904}
I20260709 13:16:16.851361 11476 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:16.852510 14393 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:16.876143 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:16.882126 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45300.9us
W20260709 13:16:16.893796 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 140434us
I20260709 13:16:16.894580 14237 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:16.908209 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.894014 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.912384 11471 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:16.895473 14237 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.879112 14482 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:16.856042 14455 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:16.939234 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 281378us
I20260709 13:16:16.915968 14330 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.947402 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:16.948256 11471 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.962080 11477 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:16.963667 11477 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:16.969003 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54488.2us
I20260709 13:16:16.970207 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:16.971170 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1934) took 1012 ms. Trace:
I20260709 13:16:16.971709 11470 rpcz_store.cc:276] 0709 13:16:15.959203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.263834 (+304631us) service_pool.cc:224] Handling call
0709 13:16:16.293678 (+ 29844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.314790 (+ 21112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.970849 (+656059us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5400059954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:16.971091 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39336192}
I20260709 13:16:16.972834 14455 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:16.933302 14482 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:16.990087 11471 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:16.983667 14479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.035s	sys 0.036s
I20260709 13:16:16.994592 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:17.000325 11514 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 3.186s (SENT)
W20260709 13:16:16.991880 14537 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.191s	user 0.000s	sys 0.004s
I20260709 13:16:16.984440 14499 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.001832 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:17.004175 11470 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:16.982584 14330 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:17.005167 14479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.035s	sys 0.037s
I20260709 13:16:17.002460 14487 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:17.017632 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:17.017835 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:17.018462 12119 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:17.018770 12113 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:17.018524 12118 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:17.019611 14482 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.013230 14383 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.029534 12119 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:17.031019 14538 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.231s	user 0.000s	sys 0.004s
I20260709 13:16:17.034631 14360 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:16:17.034633 14286 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:16:17.037231 14286 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:17.037606 14286 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:17.038565 11476 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:17.041461 11544 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:17.041851 11542 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:17.043421 11545 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:16:17.043740 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:17.046438 14330 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:17.046813 11546 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:16:17.048856 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50136.9us
I20260709 13:16:17.050294 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:17.051369 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:17.052124 11550 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:17.053699 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.054759 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:17.055181 11544 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:17.058599 11542 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:17.063429 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:17.064059 11548 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:17.065732 11550 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:17.070832 11542 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:17.072631 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:17.073172 11547 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:17.076215 11550 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:17.064466 14482 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:17.038435 14286 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.074126 13873 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 13:16:17.080178 11514 leader_election.cc:336] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.615s (SENT)
I20260709 13:16:17.080664 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.089257 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:17.089576 11548 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:17.098603 14383 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:17.099161 14237 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:17.089257 14435 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:17.104930 11514 leader_election.cc:336] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.619s (SENT)
I20260709 13:16:17.117391 11547 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:17.124504 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:17.133589 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:17.133482 14487 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:17.134778 14303 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:17.135290 14303 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:17.135171 14480 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:17.136524 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1913) took 1257 ms. Trace:
I20260709 13:16:17.136344 14303 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.136950 11475 rpcz_store.cc:276] 0709 13:16:15.878725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.886579 (+  7854us) service_pool.cc:224] Handling call
0709 13:16:15.897276 (+ 10697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.938134 (+ 40858us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:16.102098 (+163964us) log.cc:844] Serialized 14 byte log entry
0709 13:16:16.490951 (+388853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:16.491038 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:16.491104 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.136286 (+645182us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b5400214654. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:17.136337 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.136463 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":712,"prepare.queue_time_us":400202,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":2449,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5663,"spinlock_wait_cycles":93342976,"thread_start_us":348697,"threads_started":2,"wal-append.queue_time_us":426884}
I20260709 13:16:17.145228 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:17.149602 11514 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.647s (SENT)
I20260709 13:16:17.149219 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:17.148766 14482 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.153903 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:17.144743 14500 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:17.144953 14330 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:17.150394 14297 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:16:17.163192 11442 leader_election.cc:400] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:17.164179 11442 leader_election.cc:403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:17.164767 14383 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:17.167239 14533 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.171914 14383 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:17.173766 14383 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:17.176151 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.177066 11475 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:17.178718 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.180514 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.181245 11469 raft_consensus.cc:2368] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
I20260709 13:16:17.181855 14457 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:17.183017 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.185000 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.187001 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:17.187420 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.187568 11469 raft_consensus.cc:2376] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
I20260709 13:16:17.188752 14264 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:17.189397 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.191272 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.191988 11469 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.193113 11475 raft_consensus.cc:2368] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:17.194730 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.195638 11469 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.152961 14257 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 13:16:17.208350 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66315.6us
W20260709 13:16:17.209717 11514 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.691s (SENT)
I20260709 13:16:17.210229 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.214066 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.215225 11469 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:17.217831 14554 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:16:17.228840 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:17.232152 11514 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.036s (SENT)
I20260709 13:16:17.232806 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:17.234143 11514 leader_election.cc:336] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.045s (SENT)
I20260709 13:16:17.235297 14239 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.243714 14435 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.195081 14330 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.158113 14500 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.227785 14257 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 13:16:17.262662 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.019s	sys 0.000s
I20260709 13:16:17.470269 14500 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:17.234694 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:17.422482 14520 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.314033 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.432356 14435 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.471369 14352 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:17.471261 14520 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.471750 14435 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:17.472455 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 238339us
W20260709 13:16:17.473062 11514 leader_election.cc:336] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.068s (SENT)
I20260709 13:16:17.473335 14520 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:17.473423 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.474208 14352 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.474480 14435 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:17.475716 12113 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:17.236270 14513 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:16:17.476929 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.479081 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.479704 11398 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:17.481190 14457 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:17.485952 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:17.486635 12113 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:17.443768 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:17.445994 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.487970 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:17.446784 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.488837 14520 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:17.489178 11475 raft_consensus.cc:2368] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.489203 14520 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:17.447633 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.489836 14520 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.490053 11472 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:17.447989 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.449147 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.491262 11470 raft_consensus.cc:2410] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:16:17.492168 11514 leader_election.cc:336] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.101s (SENT)
I20260709 13:16:17.449422 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.492655 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.456310 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.493417 11474 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:17.493769 14435 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:17.494192 11514 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.301s (SENT)
I20260709 13:16:17.494635 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.459120 14330 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:17.521425 14330 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:17.459632 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:17.463335 12114 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:16:17.462307 14556 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.216s	user 0.000s	sys 0.008s
I20260709 13:16:17.524223 14383 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:17.273641 14532 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:17.470358 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.019s	sys 0.000s
W20260709 13:16:17.528898 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 1.09249e+06us
W20260709 13:16:17.488178 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:17.531318 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.533809 12119 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:17.446494 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.535346 14383 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:17.536375 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.536983 11476 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:17.537752 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.491321 11477 raft_consensus.cc:2410] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:17.495195 11398 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:17.496484 14475 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.498502 14286 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:17.539669 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.540325 11477 raft_consensus.cc:2368] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:17.541021 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.541941 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:16.172075 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:17.216571 14299 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:16:17.543496 11516 leader_election.cc:400] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.543805 11516 leader_election.cc:403] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:17.544744 11516 leader_election.cc:400] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
W20260709 13:16:17.503134 14542 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.673s	user 0.010s	sys 0.006s
I20260709 13:16:17.501735 14520 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.545763 11516 leader_election.cc:403] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:17.546322 14435 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.546597 14520 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:17.547026 14435 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.508340 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.547582 11469 raft_consensus.cc:2376] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
I20260709 13:16:17.547950 11516 leader_election.cc:400] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:17.548241 11516 leader_election.cc:403] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:17.548434 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.548740 14330 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:17.549113 11469 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.508597 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.549814 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.509603 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.550467 11469 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:17.550850 11471 raft_consensus.cc:2410] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:17.549767 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:17.538666 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.551991 11476 raft_consensus.cc:2368] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.551863 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.552695 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.552803 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.553393 11474 raft_consensus.cc:2368] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.542858 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:17.555402 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.550047 12118 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
W20260709 13:16:17.503250 14546 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.496s	user 0.006s	sys 0.000s
I20260709 13:16:17.510692 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.557088 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.557468 11470 raft_consensus.cc:2368] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:16:17.511603 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.511631 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.511945 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.511962 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.430461 14533 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:17.515918 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.569177 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:17.520522 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.544479 14297 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:17.558872 11473 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:17.585984 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.586784 11402 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:17.450042 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 198634us
I20260709 13:16:17.556596 11477 pending_rounds.cc:85] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 2
I20260709 13:16:17.556866 14330 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:17.569561 11468 raft_consensus.cc:2368] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:17.571183 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.569255 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.593219 11475 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.558190 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.573061 11474 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:16:17.573079 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:17.573714 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.594812 11469 raft_consensus.cc:2368] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 3. Current term is 4.
W20260709 13:16:17.517326 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85853.8us
W20260709 13:16:17.582113 11514 leader_election.cc:336] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.316s (SENT)
I20260709 13:16:17.589596 11477 pending_rounds.cc:107] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:16:17.595772 11477 raft_consensus.cc:2889] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.598516 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.598783 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.598446 11470 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:16:17.603575 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.603909 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.604775 11469 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:17.605075 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.605615 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.607262 11469 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:17.609066 11474 pending_rounds.cc:85] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 8
I20260709 13:16:17.609525 11474 pending_rounds.cc:107] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:16:17.609833 11474 raft_consensus.cc:2889] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.611390 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.612162 11475 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:17.612234 11471 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:17.614373 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.595105 14239 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:17.616180 14239 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.618950 14239 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:17.620155 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.621434 11471 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:16:17.623493 11471 pending_rounds.cc:85] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:16:17.623833 11471 pending_rounds.cc:107] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:16:17.624189 11471 raft_consensus.cc:2889] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.626147 11475 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
I20260709 13:16:17.626948 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.639693 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.640930 11393 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:17.644169 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.644286 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:17.645028 11471 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:17.646818 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.647857 11393 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:17.652253 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.654886 11543 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:17.655606 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.655913 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:17.660131 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.660444 11402 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 8. Ops: [6.4-6.4]
I20260709 13:16:17.662385 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.664094 11549 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:16:17.664487 11549 pending_rounds.cc:85] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 6
I20260709 13:16:17.664710 11549 pending_rounds.cc:107] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:16:17.664937 11549 raft_consensus.cc:2889] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.597805 14383 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:17.667893 11543 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:16:17.668042 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:17.669211 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:17.660804 11401 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:17.637583 14239 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:17.689635 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:17.660818 14532 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:16:17.699715 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.699337 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.701591 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.698668 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:17.698247 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.700490 11397 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:17.725970 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.638962 14299 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:16:17.665963 14488 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:16:17.687474 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:17.693621 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:17.736364 11548 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:17.679483 14448 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:17.739130 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 51739.6us
I20260709 13:16:17.693802 13873 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.758960 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 69450.7us
I20260709 13:16:17.694020 14437 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:17.759989 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:17.761438 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:17.765822 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.760385 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:17.766940 11542 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:17.767102 11544 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:17.678485 14461 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:16:17.770951 11476 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:17.773296 11471 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:17.777173 11471 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:17.684489 14543 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:17.785701 14488 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:17.666311 14383 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.786665 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:17.792467 11471 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:17.786621 14352 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:17.769644 14455 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:17.845788 11400 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:17.726629 14545 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:16:17.759464 13873 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:17.795109 14448 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:17.891435 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:17.893123 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 53921.8us
W20260709 13:16:17.993026 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.893245 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:17.787391 14257 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:16:17.788635 14303 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 13:16:17.830900 14352 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:17.787168 14264 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:16:17.788429 14487 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:16:17.789027 14262 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:16:17.831960 14455 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:17.854146 14543 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:17.907940 14478 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.940888 14564 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.950462 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.951876 14383 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:17.958791 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 257936us
I20260709 13:16:17.924134 14479 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.975327 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:17.986104 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 403999us
W20260709 13:16:18.039431 11514 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.339s (SENT)
I20260709 13:16:17.917326 14482 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:18.017802 11298 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 243) took 24704 ms (client timeout 15000 ms). Trace:
I20260709 13:16:18.025983 12119 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:16:18.041648 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 219847us
I20260709 13:16:18.028959 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.017491 14352 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:18.049307 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:17.995743 14437 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:18.031208 14455 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:18.062577 11442 connection.cc:256] RPC call timeout handler was delayed by -2.4353530809999029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 13:16:18.063179 11442 connection.cc:256] RPC call timeout handler was delayed by -2.3552530810000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:16:18.063501 11442 connection.cc:256] RPC call timeout handler was delayed by -2.3291530809999585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
W20260709 13:16:18.063819 11442 connection.cc:256] RPC call timeout handler was delayed by -2.2490530810000564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:16:18.064141 11442 connection.cc:256] RPC call timeout handler was delayed by -2.1644530809999196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 13:16:18.064503 11442 connection.cc:256] RPC call timeout handler was delayed by -2.1293530809998629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 13:16:18.064857 11442 connection.cc:256] RPC call timeout handler was delayed by -1.9610530810000455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:16:18.065213 11442 connection.cc:256] RPC call timeout handler was delayed by -1.9448530810000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:16:18.065551 11442 connection.cc:256] RPC call timeout handler was delayed by -1.8863530809999247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
I20260709 13:16:18.084573 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.179049 11550 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:18.184892 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:18.189510 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.030390 14491 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.152s
I20260709 13:16:18.176007 14573 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.040663 14478 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.051262 14372 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.162s
I20260709 13:16:18.049342 14482 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:18.274000 14570 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:16:18.251642 14565 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:16:18.212329 14504 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:18.346199 14568 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.322s	user 0.000s	sys 0.007s
W20260709 13:16:18.345568 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 319699us
I20260709 13:16:18.200414 14574 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.237181 14579 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.132s
I20260709 13:16:18.290838 14583 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 13:16:18.356861 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:18.345119 14542 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.053s	user 0.002s	sys 0.006s
W20260709 13:16:18.358496 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 274017us
W20260709 13:16:18.043164 11298 rpcz_store.cc:269] 0709 13:15:53.286953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.287570 (+   617us) service_pool.cc:224] Handling call
0709 13:15:53.988294 (+700724us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:17.975963 (+23987669us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:14.634818 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:14.647563 (+ 12745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:14.647611 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:16:14.647644 (+    33us) tablet.cc:662] Decoding operations
0709 13:16:14.648768 (+  1124us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:14.648846 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:14.648874 (+    28us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:16:14.649790 (+   916us) tablet.cc:701] Row locks acquired
0709 13:16:14.649844 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:16:14.650030 (+   186us) write_op.cc:270] Start()
0709 13:16:14.650183 (+   153us) write_op.cc:276] Timestamp: P: 1783602974649963 usec, L: 0
0709 13:16:14.650205 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:16:14.689182 (+ 38977us) log.cc:844] Serialized 3990 byte log entry
0709 13:16:14.867241 (+178059us) op_driver.cc:464] REPLICATION: finished
0709 13:16:14.877140 (+  9899us) write_op.cc:301] APPLY: starting
0709 13:16:14.877316 (+   176us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:14.877627 (+   311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:14.877808 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:15.029648 (+151840us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:15.029979 (+   331us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:16:15.030038 (+    59us) write_op.cc:312] APPLY: finished
0709 13:16:15.030378 (+   340us) log.cc:844] Serialized 77 byte log entry
0709 13:16:15.030573 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:15.030694 (+   121us) write_op.cc:454] Released schema lock
0709 13:16:15.030756 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:15.031209 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:15.031236 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":110696,"reported_tablets":50,"spinlock_wait_cycles":1078201088,"child_traces":[["op",{"apply.queue_time_us":9441,"apply.run_cpu_time_us":4616,"apply.run_wall_time_us":154100,"num_ops":10,"prepare.queue_time_us":739085,"prepare.run_cpu_time_us":8014,"prepare.run_wall_time_us":124719,"raft.queue_time_us":64520,"raft.run_cpu_time_us":9702,"raft.run_wall_time_us":9725,"replication_time_us":216838,"thread_start_us":693282,"threads_started":4,"wal-append.queue_time_us":75775,"wal-append.run_cpu_time_us":17154,"wal-append.run_wall_time_us":1296146}]]}
I20260709 13:16:18.376706 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:18.377050 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.287612 14581 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.046706 14564 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.046066 14479 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.367302 11550 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:16:18.290966 14586 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:16:18.289803 14543 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.370271 13873 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:18.372095 14577 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:16:18.340514 14572 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.379446 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.393589 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:18.386482 11514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.068s (SENT) [suppressed 56 similar messages]
W20260709 13:16:18.382141 14578 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.127s	user 0.000s	sys 0.007s
I20260709 13:16:18.372962 14597 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.350929 14573 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.418386 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.419713 11545 raft_consensus.cc:2410] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:16:18.405455 14504 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.380560 14566 raft_consensus.cc:2764] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:16:18.249228 14448 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.417384 11544 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:18.407563 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:18.345158 14437 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.375582 14574 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.429663 14587 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.384194 14581 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.453217 14598 raft_consensus.cc:2764] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:16:18.408610 14572 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.447824 14600 raft_consensus.cc:2764] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 13:16:18.466440 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 73001.3us
I20260709 13:16:18.473533 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.456974 14461 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.423676 14597 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:18.479241 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 90445.4us
I20260709 13:16:18.488675 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.496862 14352 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.503319 14608 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.503913 14608 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.506062 14608 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.506356 14478 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.508047 11545 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:18.498441 14582 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:16:18.499312 14572 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.480893 14461 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.497536 14504 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.507278 14352 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:18.516289 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.487455 14601 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.463538 14587 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.491525 14600 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:18.521404 14479 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.523064 14504 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.523352 14615 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.524000 14615 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.523859 14504 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.526329 14615 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.499338 14564 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.527341 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.528744 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:18.530030 11514 leader_election.cc:400] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9.
I20260709 13:16:18.506949 14455 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:18.519183 14591 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:16:18.531158 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.534432 14581 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:18.534312 11514 leader_election.cc:400] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.535588 14574 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.535691 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:18.536262 14574 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:18.536636 14574 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.537004 14574 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:18.537122 11514 leader_election.cc:400] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.537215 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 466) took 1004 ms. Trace:
I20260709 13:16:18.538487 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.540222 14597 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.540822 14564 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.531592 14352 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.542860 11400 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:18.543740 14455 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.521409 14573 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:18.523279 14611 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:16:18.545208 14602 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:18.545769 14602 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.546188 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.547463 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.519636 14572 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.548385 14461 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:18.549769 14561 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.629s	user 0.003s	sys 0.000s
I20260709 13:16:18.519050 14612 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.551270 14612 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.553387 14612 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:18.559906 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 591) took 1067 ms. Trace:
I20260709 13:16:18.560351 12113 rpcz_store.cc:276] 0709 13:16:17.492239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.495610 (+  3371us) service_pool.cc:224] Handling call
0709 13:16:17.495951 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.496342 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.496377 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.496864 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:16:17.637823 (+140959us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:17.637872 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.637967 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.548989 (+911022us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:18.559568 (+ 10579us) raft_consensus.cc:1685] finished
0709 13:16:18.559593 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.559844 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":5641984,"thread_start_us":140491,"threads_started":1,"wal-append.queue_time_us":335807}
I20260709 13:16:18.548405 14572 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.564035 11442 leader_election.cc:400] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:18.563230 14601 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:18.567174 14601 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:18.567955 14601 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.569124 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.569466 14572 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.546243 14602 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:18.572237 14562 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.754s	user 0.000s	sys 0.001s
I20260709 13:16:18.574223 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 489) took 1047 ms. Trace:
I20260709 13:16:18.574582 12114 rpcz_store.cc:276] 0709 13:16:17.526499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.527497 (+   998us) service_pool.cc:224] Handling call
0709 13:16:17.527813 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:17.528153 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.528181 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:17.528770 (+   589us) log.cc:844] Serialized 52 byte log entry
0709 13:16:17.604872 (+ 76102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:17.604908 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.604979 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.571578 (+966599us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:18.573929 (+  2351us) raft_consensus.cc:1685] finished
0709 13:16:18.573950 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.574167 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":166034176,"thread_start_us":75687,"threads_started":1,"wal-append.queue_time_us":159872}
I20260709 13:16:18.576028 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.577469 14581 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:18.578128 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.578922 14597 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.578131 14581 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:16:18.537946 11468 rpcz_store.cc:276] 0709 13:16:17.532892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.607692 (+ 74800us) service_pool.cc:224] Handling call
0709 13:16:17.651248 (+ 43556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.876280 (+225032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.876313 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.876357 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:17.876393 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.536918 (+660525us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400059e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:18.536980 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.537140 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468124928}
I20260709 13:16:18.526734 14504 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.589643 11399 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:18.590517 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.595991 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.595870 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.597886 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:18.599167 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:18.599556 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.599896 11472 raft_consensus.cc:2368] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.600175 11551 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:18.600731 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.601344 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.601998 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.603483 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.570264 14572 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:18.606859 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:18.611196 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 72778.8us
I20260709 13:16:18.612363 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.608333 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.616641 14601 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:18.617161 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.619196 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.610423 11544 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:16:18.622306 11514 leader_election.cc:400] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.624763 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.627156 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.602806 14581 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:18.628554 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.654484 12114 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44539667cbc04657b801ff659dee5bde"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:56142
I20260709 13:16:18.627950 14608 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.669194 12114 raft_consensus.cc:606] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:18.667606 14383 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:18.659480 14594 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.672451 14383 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.610407 14597 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:18.662755 14602 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.652117 14330 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:16:18.623585 14615 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:18.674913 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47860.6us
I20260709 13:16:18.669934 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.606624 14479 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.607419 14461 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:16:18.604743 14574 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.675825 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.624024 14435 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.623466 14601 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.677523 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.627982 14600 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.673825 14383 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:18.677326 14513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.002s	sys 0.007s
I20260709 13:16:18.674068 14605 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:18.627741 14239 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:16:18.695325 14597 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.696076 14597 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.697017 14602 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.698680 14597 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.698843 14574 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.699967 14476 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.700578 14476 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.701685 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.702489 14573 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:18.702898 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.703174 14330 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:18.703552 14330 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.703912 14330 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:18.704062 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.705255 11514 leader_election.cc:400] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.705300 14352 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.706104 14352 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:18.706941 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.707432 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.707569 11473 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:18.708158 14603 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.684579 14581 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:18.708484 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:18.708691 14581 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.679684 14237 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:18.709101 14581 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.709220 14237 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.709620 14237 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.708381 14352 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.711342 14352 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:18.706255 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:18.717518 14605 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:18.718292 14482 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.719226 14605 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:18.719470 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:18.719657 14605 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:18.720114 12113 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:18.720479 14605 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.721122 14476 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.722185 14603 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.723007 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.723729 11544 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:18.725144 12113 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:18.725826 14603 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:18.726313 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.729765 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.730695 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.731526 14605 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:18.732647 14581 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.734313 14596 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.734859 14596 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.735589 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:18.736141 11545 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.736928 14596 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.711323 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.737512 14596 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:18.737864 11468 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:18.741483 14594 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:18.741775 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.741950 14594 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.742413 14594 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:18.743532 14584 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.390s	user 0.000s	sys 0.003s
I20260709 13:16:18.737900 14596 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.717935 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:18.746886 11545 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:18.748162 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.748255 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:18.752185 12114 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:18.754215 14604 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:18.754838 14604 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.759804 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:18.760593 12113 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
W20260709 13:16:18.686501 14513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.002s	sys 0.007s
I20260709 13:16:18.763446 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:18.766513 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:18.737432 14573 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.780114 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:18.786844 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 417282us
I20260709 13:16:18.753587 14596 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:18.762377 14597 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:18.756497 14435 raft_consensus.cc:2764] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:16:18.679162 14479 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.798314 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:18.799441 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:18.764381 14615 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:18.804469 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 45787.9us
I20260709 13:16:18.805619 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:18.751317 14330 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.776849 14603 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:18.862758 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 82756us
W20260709 13:16:18.856667 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45738.1us
I20260709 13:16:18.863662 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.869438 11514 leader_election.cc:400] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
I20260709 13:16:18.764605 14383 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.869338 11545 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
W20260709 13:16:18.875610 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 67866.9us
W20260709 13:16:18.877844 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 81344.6us
I20260709 13:16:18.794266 14587 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:18.908995 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 97007.2us
I20260709 13:16:18.792259 14595 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:18.915863 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:18.921877 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 92565.9us
I20260709 13:16:18.918409 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.807226 14239 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.922720 11471 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:18.919968 12119 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:18.856050 14602 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:18.921003 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.793877 14491 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:16:18.857220 14581 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:18.935304 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:18.936084 11393 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:18.936671 11399 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:18.937762 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.939127 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.939239 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.940696 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.941252 11550 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:18.941347 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.942184 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.942607 11550 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:18.944180 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.945127 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.951823 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:18.938541 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.951673 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:18.918879 14476 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:18.954648 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.956351 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:18.957525 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:18.957818 14563 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:18.962123 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.930788 14587 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.958232 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.959182 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1970) took 1282 ms. Trace:
I20260709 13:16:18.959403 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:18.959903 11397 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:18.967360 11544 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:18.959976 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.967231 11543 rpcz_store.cc:276] 0709 13:16:17.677103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.677529 (+   426us) service_pool.cc:224] Handling call
0709 13:16:17.677938 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.578210 (+900272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.578250 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.578720 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.651902 (+ 73182us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:18.651962 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.945816 (+293854us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54003ea3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:18.945867 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.945924 (+    57us) raft_consensus.cc:1685] finished
0709 13:16:18.945936 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.959098 (+ 13162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5958,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":2069567872,"thread_start_us":72748,"threads_started":1,"wal-append.queue_time_us":112982}
I20260709 13:16:18.879256 14597 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:18.958165 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.964382 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:18.965948 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.966224 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.971272 11471 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:18.971376 11547 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:18.868336 14573 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
W20260709 13:16:18.971851 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 183008us
I20260709 13:16:18.954105 14435 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:18.961319 14545 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:18.977435 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 172218us
I20260709 13:16:18.980504 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:18.962280 14476 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.967706 14600 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:18.978945 14571 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 13:16:18.983958 14621 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:16:18.984472 14383 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.979548 14543 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:18.970109 14587 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:18.986093 14476 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:18.987903 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:18.989738 14597 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:18.982084 14603 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:19.002511 14603 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:19.003517 14603 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.004751 14603 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.006548 14587 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:18.978958 14626 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:19.007428 11475 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:19.007978 14360 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:19.008368 14360 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.008679 14360 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:19.010078 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:18.992063 14330 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:18.934584 14239 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.011242 14626 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.011677 14626 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.012868 14239 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.013228 14604 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.014572 14479 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.016850 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:19.019469 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43915.8us
I20260709 13:16:19.020727 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.993530 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.022431 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:18.990665 14352 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.000936 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.001152 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:18.931641 14595 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.021333 11548 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:19.022917 11543 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:18.995944 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:19.017640 11475 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:18.993101 14435 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:18.995518 14600 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.024288 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.026960 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:19.032948 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:19.038113 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.033772 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.036478 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:18.997910 14602 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:19.042039 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46136.5us
I20260709 13:16:19.046008 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.046716 11475 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:19.044188 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:19.048826 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.041494 14600 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.045972 14461 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:19.051360 14461 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.051699 14461 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.059201 14352 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.060480 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.061724 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.063956 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1978) took 1268 ms. Trace:
I20260709 13:16:19.064363 11542 rpcz_store.cc:276] 0709 13:16:17.795529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.049671 (+254142us) service_pool.cc:224] Handling call
0709 13:16:18.591368 (+541697us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.591707 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.591737 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.606811 (+ 15074us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.758702 (+151891us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:18.758760 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.758848 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.011048 (+252200us) raft_consensus.cc:1685] finished
0709 13:16:19.011068 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.063688 (+ 52620us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54000b1b50. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:19.063895 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":518,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7038,"spinlock_wait_cycles":949849856,"thread_start_us":91394,"threads_started":1,"wal-append.queue_time_us":296701}
I20260709 13:16:19.066337 11300 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 4 to 5, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 5 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.068001 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.069665 11473 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:16:19.069427 14435 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:19.070286 14628 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:16:19.071794 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.077636 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.079932 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.070088 11300 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 3 to 4, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 4 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.042739 14239 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:19.101701 11546 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:19.103158 14522 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:19.103633 14522 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.103976 14522 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.104061 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.104775 11401 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:19.106063 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.107573 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.107445 14360 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.108309 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.109587 14360 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:19.112188 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:19.035629 14237 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.113560 11300 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 2 to 3, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.117228 11300 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 1 to 3, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 3 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:19.118541 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:19.121563 14530 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:19.123252 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.123665 14543 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:19.123979 11400 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:19.124167 14572 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:19.122644 11300 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 1 to 2, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 2 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.124624 14572 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:19.073028 14476 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.125519 14572 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.125672 11300 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 2 to 3, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 3 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:19.126545 14572 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.127368 11300 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 4 to 5, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 5 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:19.127766 14476 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.128712 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.129374 11396 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:19.129129 11300 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 3 to 4, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 4 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.130548 14239 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.133350 14594 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.135533 14594 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:19.135766 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99384.5us
I20260709 13:16:19.136135 14594 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:19.136549 14594 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.136914 14594 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.136744 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:19.046162 14602 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:19.141196 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 84960.1us
I20260709 13:16:19.141690 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:19.142308 11542 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:19.124255 14543 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:19.148120 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:19.130971 14239 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.153824 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:19.159363 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.113444 14596 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.168673 11476 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:19.172463 11516 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.173765 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:19.176398 12083 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:19.177536 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.200572 11545 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.162590 14598 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:16:19.159804 14601 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:19.163278 14605 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:19.187315 14602 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.189386 14600 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.171064 14461 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.172508 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:19.210012 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:19.194314 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:19.196250 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:19.201143 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.170930 14566 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.175333 14513 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.210807 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:19.215255 11542 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:19.211886 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.219534 11516 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.221581 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:19.194604 14435 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.202210 14573 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.231956 11470 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:19.206144 14605 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.252724 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.215830 14299 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.295338 14299 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.297657 14299 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.281677 11545 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:19.286815 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 67376.4us
I20260709 13:16:19.299065 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.300336 14649 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:19.301219 11516 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:19.215067 14383 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:19.223290 14478 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:19.222208 14602 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:19.222919 14352 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:19.224180 14601 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:19.210316 14616 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:19.304102 11474 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:19.312328 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.312866 11471 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.304587 14239 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.302541 11396 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:19.303098 14330 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:19.302968 14564 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:19.303973 14605 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.217284 14596 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.303758 14604 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:19.268361 14571 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.293345 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:19.341944 11474 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:16:19.348137 14479 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:19.348659 14479 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.349037 14479 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.351938 14461 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.357837 12116 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:19.305619 14649 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.226464 14513 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.302811 14435 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.220642 14566 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.361310 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.363332 11471 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:19.371760 11550 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:19.310197 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 88710.7us
I20260709 13:16:19.368959 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:19.372936 12118 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:19.213243 14543 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.313741 14616 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.377571 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.355291 14602 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:19.376157 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:19.368391 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 67240us
I20260709 13:16:19.368096 14587 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:19.382232 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 175766us
I20260709 13:16:19.376286 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:19.383510 12083 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:19.378173 11551 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:19.389909 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:19.367821 14461 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:16:19.392930 11550 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:19.379258 14616 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.389168 14545 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:19.370209 14649 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.391388 14543 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.398620 14654 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:19.403549 14573 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:19.407611 14638 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:19.367750 14571 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.383173 14595 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:19.411401 11551 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:19.390869 14601 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:19.416241 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.394129 14478 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:19.420277 14647 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.421177 14239 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.381686 14626 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.451782 14650 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.460661 14650 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.463137 14478 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:19.464450 14478 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.409178 14600 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.459089 14601 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:19.467540 14647 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.468773 14601 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.448917 14612 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.455360 14626 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:19.412189 14545 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.471262 14545 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.449425 11550 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.477919 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.478576 11477 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.458550 14571 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.479943 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.480737 14594 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.482138 11472 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:19.470180 12118 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:19.488140 14513 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:19.480777 11475 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:19.507684 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:19.470736 14612 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.517206 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.522976 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.525527 11470 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:19.527787 11475 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:19.529551 11475 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:16:19.530020 11475 pending_rounds.cc:85] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:16:19.534809 11475 pending_rounds.cc:107] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:16:19.535221 11475 raft_consensus.cc:2889] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:19.529703 11550 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:19.532298 14500 raft_consensus.cc:993] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:16:19.515205 14239 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.542315 11470 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:19.522872 14612 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.514947 14545 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.548090 14543 raft_consensus.cc:1081] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:16:19.549572 14545 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.552784 11477 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:19.554450 11469 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:19.519338 14650 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.519580 14649 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.564610 14479 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.576493 14605 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.578429 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:19.578707 14605 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.579421 12114 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:19.580364 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.582741 11475 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:19.584658 11542 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.589104 14659 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:19.590526 14545 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:19.591154 14545 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:19.592092 14545 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.593802 14435 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.594632 14437 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:19.595744 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.596830 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.597460 11471 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:19.597280 14348 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:19.602216 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.603404 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:19.605455 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.509716 14522 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.608556 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.607097 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:19.610697 11475 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:19.613642 14372 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:19.621384 14372 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:19.621629 14479 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.622262 14372 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.623538 14372 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.624632 14348 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:19.625074 14348 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:19.625869 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:19.626659 14513 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:19.627039 14513 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.627450 14513 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.609396 14257 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:16:19.630069 14616 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:19.625869 14348 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.633136 14348 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:19.633342 14623 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.765s	user 0.004s	sys 0.000s
I20260709 13:16:19.638059 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:19.640282 11542 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:19.785238 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 133575us
I20260709 13:16:19.828846 11470 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:16:19.834794 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.794683 14650 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.795754 14435 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.860993 14581 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.870182 14583 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:19.872145 11547 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.875453 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:19.877213 14237 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:19.787063 14580 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:16:19.930135 14372 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:19.930598 14372 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:19.932468 14372 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.933761 14372 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.927675 14583 consensus_queue.cc:1243] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:16:19.894100 14648 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:19.907202 14604 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:19.856125 14360 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:16:19.856343 14650 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.891587 14435 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.895032 14581 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.949780 11547 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:19.938606 14648 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:19.977185 14648 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.984880 14566 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:19.985801 14435 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:19.946062 14604 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:20.002465 14522 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.097105 14435 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.109644 14435 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.017400 14604 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:20.123279 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 247897us
I20260709 13:16:20.127010 14650 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.128100 14615 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:20.128719 14615 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:20.129959 14615 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.132050 14435 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.132309 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.142055 14615 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.142354 14649 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:20.142912 14649 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:20.143810 14649 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.146838 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.147706 14435 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:20.147897 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:20.148172 14435 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.148614 14435 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.150698 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.151628 11551 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:20.150647 11548 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:16:20.141131 14513 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:16:20.154026 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.154335 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.159464 14237 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:20.160087 14237 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:20.161044 14237 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.163353 14581 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.163708 14616 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.165206 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1320) took 1252 ms. Trace:
I20260709 13:16:20.165602 11476 rpcz_store.cc:276] 0709 13:16:18.912970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.913089 (+   119us) service_pool.cc:224] Handling call
0709 13:16:19.071471 (+158382us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:20.164984 (+1093513us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b540044bbd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:20.165127 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228144256}
I20260709 13:16:20.167143 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.155138 11468 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:20.150878 11549 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:16:20.151664 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.152339 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.169512 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.170353 11550 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:20.170260 11542 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:20.132692 14533 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:20.172983 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.197129 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.140342 14594 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:20.175009 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.194803 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.200052 11542 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:20.168470 14605 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:20.203800 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.204808 11542 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:20.195574 14533 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.201756 14566 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:16:20.203419 14581 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:20.211485 14581 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.211891 14581 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:20.199040 14239 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:20.212379 14239 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.212795 14239 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:20.204569 14630 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:20.207228 14360 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:16:20.215480 14630 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.217629 14630 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.169602 14513 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.209190 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:20.219902 14513 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.220398 14630 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:20.220965 14630 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.222076 14630 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:20.224186 14658 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.719s	user 0.000s	sys 0.010s
I20260709 13:16:20.231400 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.232105 11547 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:20.235675 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.242461 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.242777 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.243319 11550 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.243386 11551 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.237078 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.247386 14601 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:20.247855 14601 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.248241 14601 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.249840 14435 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.254472 14587 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.255002 14587 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.255077 14682 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:20.255511 14682 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.255863 14682 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.279526 11542 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:20.280800 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:20.280840 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:20.282068 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:20.290933 12119 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:20.277849 14335 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:20.292302 14335 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.290980 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:20.296149 14647 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.296958 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.299976 14592 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:20.300549 14592 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.301290 14239 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.303058 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.304256 14651 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:20.304813 14651 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.305743 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 595) took 1188 ms. Trace:
I20260709 13:16:20.306114 12114 rpcz_store.cc:276] 0709 13:16:19.117512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.119083 (+  1571us) service_pool.cc:224] Handling call
0709 13:16:19.408042 (+288959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.584155 (+176113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.304900 (+720745us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540047ca54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:20.305568 (+   668us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540047ca50. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:20.305688 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057473152}
I20260709 13:16:20.307828 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:20.294734 14335 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.322284 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.322435 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.322410 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.322156 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.327659 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.332383 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:20.335088 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.330636 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.398615 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:20.399596 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:20.343412 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.401777 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:20.316445 14475 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:20.325522 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.343736 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.326372 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1984) took 1522 ms. Trace:
I20260709 13:16:20.341374 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.339890 11550 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:20.403713 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.409866 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.419674 12114 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:20.424185 12119 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:20.379199 11472 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:16:20.376992 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.409915 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.307619 14602 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.425786 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:20.430442 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.430616 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.430809 11473 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.431658 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.431756 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.432111 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.426649 11549 rpcz_store.cc:276] 0709 13:16:18.851920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.856941 (+  5021us) service_pool.cc:224] Handling call
0709 13:16:19.864926 (+1007985us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:20.150648 (+285722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.169320 (+ 18672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.326147 (+156827us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400134050. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:20.326304 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659712128}
I20260709 13:16:20.306871 14651 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.334937 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.433969 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.436703 12114 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:20.407723 14455 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:16:20.458151 14673 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:16:20.347505 14437 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:16:20.408252 14478 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:20.401075 14303 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:20.347523 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44539667cbc04657b801ff659dee5bde"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43866
I20260709 13:16:20.469864 11475 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:20.470376 11475 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:20.435078 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1999) took 1174 ms. Trace:
I20260709 13:16:20.317126 14605 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.462963 11473 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
W20260709 13:16:20.464592 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.333935 14615 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:20.430187 14475 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:20.407142 14581 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.471853 11471 rpcz_store.cc:276] 0709 13:16:19.260289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.612340 (+352051us) service_pool.cc:224] Handling call
0709 13:16:19.612587 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.612720 (+   133us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:19.924721 (+312001us) log.cc:844] Serialized 14 byte log entry
0709 13:16:20.434726 (+510005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:20.434779 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:20.434830 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.434899 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.435025 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4034,"prepare.queue_time_us":237315,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":10098,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":797462,"threads_started":3,"wal-append.queue_time_us":302415}
I20260709 13:16:20.335961 14647 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:16:20.490938 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.492503 11550 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:20.493582 11551 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:20.498708 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.482245 14297 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:20.481598 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.478309 14478 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.510073 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.439570 14602 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.513798 11551 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:20.430408 14533 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.486620 14455 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:20.528194 11549 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:20.529757 11543 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:20.485159 14677 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.441800 14601 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.516908 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:20.524423 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.487365 14239 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.499256 14615 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.483255 14605 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.535089 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.501626 14597 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.549546 11400 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.541421 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:20.507845 14262 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:20.510948 11397 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:20.505828 14297 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.556746 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.557416 11395 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:20.557770 11551 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:20.560479 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.527815 14455 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.562500 11395 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.562588 11543 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:16:20.538461 14435 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.542490 14566 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:20.571627 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:20.568826 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.572772 11542 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:20.572983 11474 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:20.548753 14239 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.535408 14677 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.575549 11543 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:20.556031 14583 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.581601 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.601761 11549 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:20.561302 14482 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.601752 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:20.613081 14522 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:20.598248 11399 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:20.602875 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:20.511515 11475 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.620570 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:20.614403 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:20.562199 14533 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.622001 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:20.552112 14615 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.568702 14262 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.553457 14597 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:20.669052 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.654341 14651 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:20.614848 14583 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.658672 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 1987) took 1514 ms. Trace:
I20260709 13:16:20.673038 11549 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:20.673187 11474 rpcz_store.cc:276] 0709 13:16:19.144485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.369698 (+225213us) service_pool.cc:224] Handling call
0709 13:16:20.167372 (+797674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.573478 (+406106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.657618 (+ 84140us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b540044bbd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:20.658407 (+   789us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b540044bbd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:20.658594 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766356480}
I20260709 13:16:20.629797 11395 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:20.668421 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.674880 11402 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:20.599668 14594 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:20.650039 14297 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.675925 14594 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.676636 14297 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.677985 14594 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.679261 14601 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.680532 14597 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.655647 14677 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.566612 14475 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.690662 14478 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.692171 14605 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.693106 14605 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.693593 14605 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.694562 14262 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.695159 14455 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.696664 14455 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:20.700721 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 80225.4us
I20260709 13:16:20.704231 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.641717 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.604063 14647 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.600678 14435 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.709074 11400 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:20.722214 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:20.723083 11470 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.725006 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.725769 11400 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:20.726941 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.731926 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.600466 14239 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:20.736462 14303 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:20.737100 14239 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.738003 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.656126 14581 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.738971 11470 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:20.667361 14682 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.740147 14435 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.741904 14682 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.660281 14587 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.743477 14435 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.744247 14587 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:20.671939 14602 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.744879 11400 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:20.728605 14592 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.717419 14601 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:20.718200 11472 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:20.617626 14522 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:20.685936 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:20.603935 14680 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:16:20.747242 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.748817 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:20.815495 11474 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:20.748997 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.817152 11468 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
W20260709 13:16:20.797000 14237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.012s	sys 0.024s
I20260709 13:16:20.750069 11551 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:20.751197 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.826910 11477 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:20.752197 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:20.761337 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:20.751155 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.615790 14437 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:16:20.810189 11475 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:20.810223 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:20.811075 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 125253us
W20260709 13:16:20.818100 12113 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.022s
I20260709 13:16:20.749895 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.831836 11476 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:20.814076 14522 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:20.787901 14602 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.825064 11474 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:20.830420 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.832784 11468 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:20.832788 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71572.9us
W20260709 13:16:20.751995 14525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.005s	sys 0.018s
W20260709 13:16:20.833828 12113 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.000s	sys 0.022s
I20260709 13:16:20.815816 14601 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:20.815201 14649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.015s	sys 0.028s
I20260709 13:16:20.843281 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:20.837127 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.817255 14659 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:20.845109 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.845688 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:20.826618 14237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.012s	sys 0.024s
I20260709 13:16:20.847661 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:20.855721 14504 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:20.857254 11545 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:20.858251 14504 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:20.859850 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.861295 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:20.854193 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.873174 14435 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:20.873529 14237 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:20.873915 14435 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.874430 14237 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.874507 14435 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.875134 14237 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:20.877931 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 443657us
I20260709 13:16:20.879068 12083 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.884292 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:20.888677 11474 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:20.895625 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.029s
W20260709 13:16:20.896025 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.005s	sys 0.029s
W20260709 13:16:20.898188 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.837783 14608 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:20.901461 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:20.848575 14525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.005s	sys 0.018s
I20260709 13:16:20.919766 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.921396 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:20.922979 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:20.905452 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.926183 14372 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:16:20.853523 14605 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:20.900768 14239 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:20.942014 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.944852 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.945178 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:20.945705 11402 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:20.945178 12112 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.019s
W20260709 13:16:20.946429 12112 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.015s	sys 0.019s
I20260709 13:16:20.946478 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.948359 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.893770 14504 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:20.931447 14262 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:20.950803 14239 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.959635 14239 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:20.921077 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.970715 11393 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:16:20.969072 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48069.4us
I20260709 13:16:20.974825 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:20.977989 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.849099 14602 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.927392 14448 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:20.983032 11402 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:16:20.848686 14649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.016s	sys 0.028s
I20260709 13:16:20.970530 14372 consensus_queue.cc:1243] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:20.987123 14262 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:20.988169 14689 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:20.988387 14262 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:20.990321 14595 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:20.983517 11402 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.991755 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:20.952103 14605 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:20.929551 14608 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:20.990732 14595 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:20.992502 11402 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:21.008412 14688 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:16:21.009083 11402 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:20.983275 14592 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:20.985141 14583 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:21.032434 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.015s	sys 0.030s
W20260709 13:16:21.034543 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.016s	sys 0.030s
W20260709 13:16:21.036971 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56508.5us
I20260709 13:16:20.980990 11398 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:21.016060 14592 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:21.047945 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.013s	sys 0.020s
W20260709 13:16:21.048336 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.013s	sys 0.020s
W20260709 13:16:21.048595 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 516518us
I20260709 13:16:21.011965 14605 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:21.092237 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46208.8us
W20260709 13:16:21.051105 14630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.019s	sys 0.020s
W20260709 13:16:21.094008 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44687.7us
I20260709 13:16:20.959970 14504 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.050029 14592 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:21.106155 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:21.026913 14601 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.094096 14697 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.092993 14615 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:21.094305 14630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.020s	sys 0.020s
W20260709 13:16:21.112537 14686 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:16:21.129361 11396 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:21.108676 14696 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.096828 14501 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:16:21.085803 14595 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.124271 14649 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:21.126052 11473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.014s
I20260709 13:16:21.126595 14615 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.143035 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:21.146885 11473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.007s	sys 0.014s
I20260709 13:16:21.152282 11475 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:21.153198 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 258350us
I20260709 13:16:21.164834 11546 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:21.107267 14608 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.133352 14697 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.180441 11402 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:21.163174 14491 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.010s	sys 0.017s
I20260709 13:16:21.141273 14696 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.164278 14487 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:16:21.170493 14695 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:21.182734 11550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.010s	sys 0.038s
I20260709 13:16:21.173584 14595 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:21.190765 11550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.010s	sys 0.038s
I20260709 13:16:21.151998 14649 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.180789 14615 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.169453 14303 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:16:21.186177 14482 consensus_meta.cc:287] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:16:21.198958 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:21.187161 14697 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.169742 14630 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:16:21.211952 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:21.212936 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:21.184867 14659 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:21.219432 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 244707us
I20260709 13:16:21.229827 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:21.231148 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 334944us
I20260709 13:16:21.232625 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:21.233433 11475 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:21.233939 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.235096 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.201362 14648 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:21.220167 14685 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:21.196560 14545 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:16:21.268399 14691 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:16:21.217746 14682 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:16:21.194305 14695 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.252168 14605 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.201783 14615 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.268812 14701 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.269266 14685 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:21.221817 14697 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.319787 14682 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:21.321300 14722 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:21.325014 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.325546 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:21.296916 14717 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:16:21.327379 14237 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.329576 14435 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.330811 14602 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.330976 14297 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:21.330875 14487 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:21.332934 14685 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:21.315425 14659 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.334107 14701 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.267715 14648 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:21.336472 14692 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 13:16:21.330802 14696 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.318120 14239 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:21.369388 14704 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:16:21.374202 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 139182us
I20260709 13:16:21.389627 11516 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.348992 14264 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.355473 14722 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.350898 14297 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:21.353838 14487 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:21.354488 14602 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:21.386063 14482 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.343667 14682 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:21.405606 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 240833us
W20260709 13:16:21.406044 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 151139us
W20260709 13:16:21.401274 14522 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.001s
I20260709 13:16:21.383235 14581 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:21.396790 11516 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.395747 14696 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.394956 14719 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.424914 11472 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:21.429073 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 272416us
I20260709 13:16:21.435542 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:21.436411 12114 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:21.436586 11468 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:21.436873 12114 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:21.437994 12114 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.438862 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.407109 14533 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:21.460390 12114 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:21.465600 14648 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.484171 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:21.424525 14649 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:21.446487 14237 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.415764 14602 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.422080 14722 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:21.523530 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.425391 14522 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.511436 12114 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:21.446116 14482 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:21.519675 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.476781 14721 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.476490 14616 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:16:21.508443 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78350.7us
I20260709 13:16:21.509016 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.527349 14491 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.531761 14491 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.426196 14581 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:21.526824 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 79434.4us
I20260709 13:16:21.537138 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.537813 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.544037 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.545094 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.485067 14487 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.480461 14297 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.466913 14648 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:16:21.547978 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.550233 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:21.550458 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.555029 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.558543 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 432) took 1213 ms. Trace:
I20260709 13:16:21.558740 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.560180 11397 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:21.559083 11398 rpcz_store.cc:276] 0709 13:16:20.345531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.433760 (+ 88229us) service_pool.cc:224] Handling call
0709 13:16:21.558375 (+1124615us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540035cfd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:21.558486 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844166528}
I20260709 13:16:21.540704 14721 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.569993 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:21.570423 14720 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:16:21.570744 11545 raft_consensus.cc:2393] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:21.571738 14697 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.567477 14605 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.567427 14297 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.575420 14724 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:21.547684 14615 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.553501 14648 consensus_queue.cc:1243] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:21.565703 14487 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:21.577351 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47344.7us
I20260709 13:16:21.532747 14482 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.581903 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:21.588482 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.589607 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:21.590935 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:21.599248 11547 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:21.591773 11551 raft_consensus.cc:2410] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:16:21.602264 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:21.617105 11551 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:21.550424 14729 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:16:21.623674 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:21.637051 14737 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:21.637770 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:21.644043 11545 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:21.583086 14695 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.648968 11551 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:21.654402 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51465.8us
I20260709 13:16:21.658143 14725 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:21.661113 14739 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:21.661530 14739 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:21.663551 14734 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:16:21.663161 14737 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.647456 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:21.649631 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.590656 14649 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:21.650804 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63081.9us
I20260709 13:16:21.611047 14732 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.668316 12112 raft_consensus.cc:2410] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:21.613560 14615 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:21.672587 11551 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:21.668308 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:21.676133 11547 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:21.636195 14616 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.675132 14732 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.693662 12115 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:21.678361 14721 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:21.678743 14707 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:16:21.698735 14715 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:16:21.679661 14602 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:21.680035 14718 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:16:21.705690 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:21.684993 14733 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:16:21.685549 14491 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:21.686308 14727 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:16:21.687484 14685 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.623219 14736 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:21.674405 14237 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:21.592578 14586 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:16:21.679032 14709 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:16:21.575469 14605 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.729583 14605 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.704985 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 495) took 1379 ms. Trace:
I20260709 13:16:21.730140 11469 rpcz_store.cc:276] 0709 13:16:20.325685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.325842 (+   157us) service_pool.cc:224] Handling call
0709 13:16:20.326128 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.340260 (+ 14132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.340290 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:20.689976 (+349686us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:20.690355 (+   379us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.126700 (+436345us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:21.126759 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.126865 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.698892 (+572027us) raft_consensus.cc:1685] finished
0709 13:16:21.698912 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.704922 (+  6010us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13780,"raft.queue_time_us":14260,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":419838848,"thread_start_us":435942,"threads_started":1,"wal-append.queue_time_us":698277}
I20260709 13:16:21.733505 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1756) took 1392 ms. Trace:
I20260709 13:16:21.739739 14482 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:21.706550 11547 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:21.670845 14729 consensus_queue.cc:1243] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:16:21.720888 14736 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:21.746865 14736 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:21.749691 14239 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.670215 14682 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:21.750447 14595 consensus_meta.cc:287] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
W20260709 13:16:21.684971 14647 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.964s	user 0.007s	sys 0.005s
I20260709 13:16:21.662423 14739 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.751246 14647 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.722028 14602 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.675151 14649 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.724579 14586 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:21.709622 14522 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.752282 11470 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:21.733899 11394 rpcz_store.cc:276] 0709 13:16:20.341386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.341520 (+   134us) service_pool.cc:224] Handling call
0709 13:16:20.683831 (+342311us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5800003698. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:20.684022 (+   191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.707575 (+ 23553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.707603 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:20.708191 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.175141 (+466950us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:21.179017 (+  3876us) log.cc:844] Serialized 14 byte log entry
0709 13:16:21.179227 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.179317 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.367894 (+188577us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b18002dd480. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:21.733317 (+365423us) raft_consensus.cc:1685] finished
0709 13:16:21.733337 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.733452 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54662,"prepare.queue_time_us":890,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":1919,"raft.queue_time_us":39808,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":21502,"spinlock_wait_cycles":13755904,"thread_start_us":466628,"threads_started":1,"wal-append.queue_time_us":687764}
I20260709 13:16:21.727685 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1993) took 1203 ms. Trace:
I20260709 13:16:21.756749 11545 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:21.758632 11393 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:21.759588 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.760344 12119 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:21.754948 11550 rpcz_store.cc:276] 0709 13:16:20.524414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.526573 (+  2159us) service_pool.cc:224] Handling call
0709 13:16:20.526876 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.194474 (+667598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.194505 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:21.217421 (+ 22916us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.393170 (+175749us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:21.393228 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.393327 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.727488 (+334161us) raft_consensus.cc:1685] finished
0709 13:16:21.727506 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.727627 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165706,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":55,"thread_start_us":833513,"threads_started":2,"wal-append.queue_time_us":300371}
I20260709 13:16:21.744530 14237 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.753603 14586 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:21.775570 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 119289us
W20260709 13:16:21.768077 14700 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.471s	user 0.003s	sys 0.001s
W20260709 13:16:21.768064 14712 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.423s	user 0.001s	sys 0.006s
I20260709 13:16:21.753592 14721 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:16:21.766767 14360 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:21.800614 11474 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 3. Current term is 4. Ops: []
W20260709 13:16:21.818017 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.800029 14360 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:21.766598 14237 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.796038 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:21.836424 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.816385 14729 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:21.838691 14729 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.819777 14685 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:21.841934 14487 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:21.841760 14608 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:21.839334 14237 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.840056 14581 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.851781 11550 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:21.839967 14745 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:21.847616 14487 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:21.854334 11549 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:21.856657 11549 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:21.856698 11550 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:21.855006 11400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:21.857491 11400 raft_consensus.cc:606] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:21.859723 11550 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:21.860277 11545 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:21.859297 14360 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.861265 11547 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:21.862821 11550 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:21.863430 11544 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:21.863873 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:21.864609 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.860617 11545 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:21.864751 11542 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:21.871722 11549 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:21.815758 14586 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.875795 11547 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:21.876070 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:21.877095 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.878197 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:21.872905 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:21.879415 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:21.880532 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.881091 11401 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:21.881920 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.882476 11401 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:21.888280 14435 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.878783 11400 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:21.886322 11545 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.893550 11545 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:21.895936 14592 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.787s	user 0.004s	sys 0.008s
I20260709 13:16:21.896272 14729 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.896370 14592 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.899312 14715 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:21.899533 14746 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:16:21.899865 14715 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.900813 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:21.901854 14715 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:21.902452 14715 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:21.903219 14586 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:21.902907 14715 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.904104 14487 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.905325 14715 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.905817 14729 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.906255 14592 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.906322 14729 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.902755 14736 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:21.909238 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:21.910312 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:21.912745 14736 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.859431 14608 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.920806 14729 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:21.941359 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:21.942582 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:21.943634 14703 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.749s	user 0.002s	sys 0.000s
I20260709 13:16:21.944818 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 441) took 1467 ms. Trace:
I20260709 13:16:21.945217 11397 rpcz_store.cc:276] 0709 13:16:20.477717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.579711 (+101994us) service_pool.cc:224] Handling call
0709 13:16:21.559779 (+980068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.560659 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.944621 (+383962us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b540035cfd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:21.944754 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247901568}
I20260709 13:16:21.949323 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 1989) took 1448 ms. Trace:
I20260709 13:16:21.950222 11551 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:21.949856 11548 rpcz_store.cc:276] 0709 13:16:20.500666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.500858 (+   192us) service_pool.cc:224] Handling call
0709 13:16:20.519806 (+ 18948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.520245 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.520270 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:20.544233 (+ 23963us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.083508 (+539275us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:21.083569 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.083669 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.943064 (+859395us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:21.946853 (+  3789us) raft_consensus.cc:1685] finished
0709 13:16:21.946876 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.949261 (+  2385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172561,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":3414272,"thread_start_us":475436,"threads_started":1,"wal-append.queue_time_us":600791}
I20260709 13:16:21.951895 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:21.953254 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:21.955307 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:21.956342 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:21.954641 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:21.966882 14728 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:21.968011 14742 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:21.959656 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:21.972281 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:21.949319 14685 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:21.977126 11472 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:21.954469 14581 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:21.953925 14682 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:21.953958 14647 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.959775 14744 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:21.957247 14482 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:16:21.986979 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:21.982301 14435 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:21.998728 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:21.983641 14761 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:21.980551 14608 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:21.998059 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:22.010426 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.977686 14586 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:16:22.042735 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7abd40007d144575bd9e785e609a62df"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:22.043673 11397 raft_consensus.cc:606] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:22.012132 14744 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:22.010298 14435 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.986439 14685 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:21.997556 14488 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:16:22.061965 14744 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.053848 14491 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:22.003145 14738 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:16:22.036633 14525 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:22.003430 14702 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:16:22.011468 14513 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:16:22.011232 14745 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:16:22.060967 14435 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:22.054586 14239 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.039747 13873 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:16:22.035864 14742 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:22.064568 14237 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.075659 14525 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.066325 14491 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:22.078336 14595 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.075596 14761 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:22.053678 14715 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:16:22.081317 14487 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:16:22.163355 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 204228us
I20260709 13:16:22.076771 14728 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:22.272388 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.273166 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.273790 11472 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:22.274752 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:22.275355 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.075964 14608 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:16:22.276542 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.286193 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:22.286792 11468 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:22.287297 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:22.285318 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.287900 11472 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:22.288256 11473 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:22.123615 14525 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:22.086653 14239 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.293041 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:22.298290 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.300561 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.302204 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.271703 14729 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.269098 14237 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.271533 14760 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:22.272100 14487 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:22.309156 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.276696 14602 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:22.316109 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.317559 11472 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:22.320427 14237 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:22.321197 14755 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 13:16:22.322755 14741 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:22.322645 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.310237 14771 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.323331 14759 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.323928 14608 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.324296 14762 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:22.325363 11546 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:22.290946 14766 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.329766 12119 pending_rounds.cc:85] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 7
I20260709 13:16:22.330441 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.330740 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:22.338948 12115 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:22.276454 14761 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:22.277195 14595 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:22.299598 14491 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.597836 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:22.715061 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2028) took 1856 ms. Trace:
I20260709 13:16:22.718482 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.714844 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 507) took 1281 ms. Trace:
I20260709 13:16:22.771906 11474 rpcz_store.cc:276] 0709 13:16:21.433224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.435175 (+  1951us) service_pool.cc:224] Handling call
0709 13:16:21.435525 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.801090 (+365565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.714557 (+913467us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400059e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:22.714770 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839133056}
I20260709 13:16:22.772944 11477 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:22.584978 11468 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:22.713812 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1776) took 1514 ms. Trace:
W20260709 13:16:22.550302 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.592446 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.773932 11393 rpcz_store.cc:276] 0709 13:16:21.199418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.199660 (+   242us) service_pool.cc:224] Handling call
0709 13:16:21.232073 (+ 32413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.759129 (+527056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.713491 (+954362us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400349754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:22.713742 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198363392}
I20260709 13:16:22.774979 11476 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:22.772162 12119 pending_rounds.cc:107] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:22.779422 12119 raft_consensus.cc:2889] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:22.698912 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.341751 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.775803 12112 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:22.771311 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.383602 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:22.403553 11545 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:22.391980 11473 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:16:22.775511 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 177796us
I20260709 13:16:22.787979 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:22.769749 11470 rpcz_store.cc:276] 0709 13:16:20.858678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.858789 (+   111us) service_pool.cc:224] Handling call
0709 13:16:20.859093 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.754106 (+895013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.713530 (+959424us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b540039b7d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:22.715002 (+  1472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053763968}
I20260709 13:16:22.792927 11476 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:22.846905 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.796340 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:22.945214 11468 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:22.945029 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:22.950379 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 162479us
I20260709 13:16:22.952174 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:22.959766 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:22.717511 14750 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.878s	user 0.003s	sys 0.001s
I20260709 13:16:22.710965 14630 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.501052 14726 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:22.417204 14758 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:16:22.650447 14435 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.564s	user 0.005s	sys 0.006s
I20260709 13:16:22.422613 14685 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:22.772210 14762 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:22.663692 14581 consensus_meta.cc:287] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.668s	user 0.000s	sys 0.010s
I20260709 13:16:22.618865 14774 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.406049 14237 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:22.699837 14262 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.842869 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:22.769065 14741 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:22.328511 14729 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:22.977252 11399 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:22.847924 14719 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:22.793262 14587 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.655833 14648 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.655593 14601 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.681558 14682 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.979254 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.694952 14721 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.018082 11545 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
W20260709 13:16:22.972434 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 484503us
I20260709 13:16:22.331055 14760 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:22.987142 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:22.951756 14744 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.695792 14525 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.995452 14581 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:22.651911 14239 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:22.633302 14732 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:22.793406 14723 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.204s
I20260709 13:16:22.633104 14773 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:22.382339 14602 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.792379 14608 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.026997 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:22.994714 11477 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:22.986274 14784 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:22.388386 14742 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.792783 14771 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.011745 14262 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.037899 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.069465 11396 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:22.986617 14435 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.018934 14786 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.415971 14728 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.793149 14766 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:22.846439 14759 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.045626 11393 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:22.980417 14630 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.031116 14721 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.011605 14237 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.039388 14741 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.082310 14741 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:23.040091 14239 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:23.083282 14237 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.083463 14239 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:23.083894 14239 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.084214 14239 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.084606 14721 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.051859 14595 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.055815 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.088279 11545 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:23.090607 11396 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:22.982939 14726 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:23.059020 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 106896us
I20260709 13:16:23.098757 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.100446 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:23.102013 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:22.440912 14487 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.072295 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.025317 14648 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.019207 14587 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.067566 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.024611 14744 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.030248 14682 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.024960 14601 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.019449 14719 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.011463 14774 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.056931 14784 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.093518 14721 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.094139 14760 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.039745 14762 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.095238 14737 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.106559 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.107028 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:23.048988 14773 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.086287 14787 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:23.112968 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.110245 14595 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.105861 14237 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:23.110116 14376 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.126458 11472 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:23.130662 11546 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:23.116714 14761 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.187642 11545 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:23.189047 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:23.194662 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.111723 14487 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.135149 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.202474 11548 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:23.115772 14782 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:23.122620 14784 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.140587 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1402) took 1264 ms. Trace:
I20260709 13:16:23.205953 11394 rpcz_store.cc:276] 0709 13:16:21.875985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.877719 (+  1734us) service_pool.cc:224] Handling call
0709 13:16:21.878005 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.878203 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.878227 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.696777 (+818550us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:22.697164 (+   387us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.046698 (+349534us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:23.046738 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.046803 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.140305 (+ 93502us) raft_consensus.cc:1685] finished
0709 13:16:23.140323 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.140528 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17322880,"thread_start_us":349173,"threads_started":1,"wal-append.queue_time_us":419262}
W20260709 13:16:23.207446 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.207577 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.141343 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.124475 11477 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:23.126745 14721 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.144230 14435 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.144472 14491 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:23.145226 14695 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:16:23.218586 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.146165 14262 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.148154 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.245455 11546 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:23.158082 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:23.158187 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:23.133016 14787 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.160995 14766 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.085788 14786 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.172238 14601 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.173532 14587 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.175050 14615 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:23.175148 14774 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.259869 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:23.136703 14762 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.260667 11546 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:23.207331 14760 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:16:23.177490 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 516) took 1214 ms. Trace:
I20260709 13:16:23.263132 11471 rpcz_store.cc:276] 0709 13:16:21.962741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.964633 (+  1892us) service_pool.cc:224] Handling call
0709 13:16:21.972061 (+  7428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.080203 (+108142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.080231 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:22.080276 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:22.080305 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.177249 (+1096944us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540044acd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.177310 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.177434 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248188672}
I20260709 13:16:23.264647 14738 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:23.264907 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.265120 14738 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.265586 11544 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:23.265976 14738 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.123808 14790 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:23.178431 14455 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:23.179044 14744 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.196621 14376 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.180459 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.204479 14237 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:16:23.209056 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.178797 14525 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.142578 14796 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:23.212368 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.212522 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.246855 11402 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:23.286545 11547 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:16:23.286449 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 77471.5us
I20260709 13:16:23.287426 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.288056 11547 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:23.287256 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.203089 14769 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:23.246888 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 88761.3us
I20260709 13:16:23.248494 12112 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:23.257664 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 129633us
I20260709 13:16:23.211884 14685 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.253870 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:23.176136 14785 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:16:23.266435 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.178011 14648 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.267037 14790 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.210562 14651 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.268252 14787 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.268705 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.268935 14786 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.269037 14771 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.269207 11549 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:16:23.214022 14761 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:16:23.210048 14522 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:23.272125 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99873.5us
I20260709 13:16:23.272089 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.213452 14715 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.298568 14738 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:23.137391 14595 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.204654 14513 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:16:23.215425 14782 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:23.201192 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.283196 11395 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:23.284518 11471 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:23.288851 11402 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:23.247725 14695 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:23.289924 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.291370 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.335189 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.270560 14721 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.297676 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.303428 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:23.304490 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 273612us
I20260709 13:16:23.275468 14455 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:23.275643 14615 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:23.284919 14774 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.341913 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 509) took 1014 ms. Trace:
I20260709 13:16:23.341702 14774 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.342324 12119 rpcz_store.cc:276] 0709 13:16:22.327126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.327638 (+   512us) service_pool.cc:224] Handling call
0709 13:16:22.329564 (+  1926us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:22.780645 (+451081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.780675 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:22.781234 (+   559us) log.cc:844] Serialized 52 byte log entry
0709 13:16:22.984321 (+203087us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:22.985255 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.985330 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.341692 (+356362us) raft_consensus.cc:1685] finished
0709 13:16:23.341713 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.341856 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132799,"thread_start_us":194355,"threads_started":1,"wal-append.queue_time_us":296643}
I20260709 13:16:23.313458 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.304988 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.313988 11473 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:23.306798 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:23.346761 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:23.347280 14772 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.024s	user 0.003s	sys 0.001s
I20260709 13:16:23.348464 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:23.348709 14262 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:23.349541 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.341941 14513 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:23.350291 14773 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.350409 14513 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.351296 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.351390 14513 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.351958 11393 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:23.291054 14744 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:23.352628 14513 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:23.352887 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.353188 14744 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:23.353708 11393 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:23.294523 14769 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.354514 14744 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:23.322379 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:23.354722 14648 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:23.351788 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.346633 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.355182 14648 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.355669 14648 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.349578 14803 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:23.356112 14769 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.356331 14803 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.357333 14803 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.358979 14601 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:23.210254 14731 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.360045 14731 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.355229 12116 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.356390 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.357383 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:23.362283 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.330199 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.330886 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.331271 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.330900 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.334327 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.336009 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:23.326326 14435 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.337085 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 136015us
I20260709 13:16:23.368150 11516 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.369477 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.327258 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.320411 14608 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.350512 14771 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:23.323905 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.307837 14685 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.365221 11394 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.336534 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.384611 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.354476 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.385505 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.385727 11393 raft_consensus.cc:2410] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:23.385679 12116 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:23.338421 14455 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.339422 14787 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:23.370460 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.389928 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.370718 11515 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.338689 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.391006 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.392545 11399 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:23.391134 11516 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.395077 11402 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:23.394539 11516 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.375231 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.324318 14790 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.294050 14796 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.376921 14649 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.377947 14782 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.428906 11395 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:23.379237 14759 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.380769 14595 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.381387 14726 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.382723 11401 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:23.338131 14522 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.400025 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.367902 14769 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:23.402227 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.373729 14799 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:23.393498 14695 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:23.333153 14651 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.337944 14715 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.436616 14787 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:23.380617 14771 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.412602 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:23.413050 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 86873.7us
I20260709 13:16:23.431852 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.472078 11393 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:23.415946 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.474256 12123 raft_consensus.cc:2410] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:23.428351 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.432868 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.384657 14773 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:23.372443 14682 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.375113 14435 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:23.453544 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 53591us
I20260709 13:16:23.482511 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:23.452697 14769 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.403251 14786 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:23.430301 14761 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:23.404793 14264 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.404852 14794 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:23.454710 14455 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.493564 14800 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.493824 14455 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.494109 14800 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.496392 14800 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.476540 14771 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.405746 14262 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.428077 14525 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:23.430733 14795 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:23.470093 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2042) took 1011 ms. Trace:
I20260709 13:16:23.461580 14608 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:23.393307 14237 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:16:23.478695 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45929us
I20260709 13:16:23.517817 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:23.519121 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:23.518796 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.521034 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:23.487190 14769 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.487859 14649 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:23.480353 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.489898 11547 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:23.462890 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.473191 14787 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:23.506537 11474 rpcz_store.cc:276] 0709 13:16:22.458244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.794444 (+336200us) service_pool.cc:224] Handling call
0709 13:16:22.794879 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.105127 (+310248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.105158 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.105200 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:23.105235 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.469869 (+364634us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400063e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.469916 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.470032 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713048832}
I20260709 13:16:23.488447 14736 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.488590 14761 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.449965 14615 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.528007 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.516664 14608 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.505571 14376 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:23.480065 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 140370us
I20260709 13:16:23.539434 14736 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.539567 14761 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.563077 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.526270 14587 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:23.571295 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.526881 14695 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:23.533504 14737 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:23.528172 14773 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:23.553336 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:23.535507 14787 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.504024 14264 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.548880 14608 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.507947 14728 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:23.508196 14682 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:16:23.515120 14262 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.550443 14794 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.591720 14794 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.585541 14616 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:23.586551 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:23.587345 14813 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.586467 14807 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.556492 14731 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.578433 14695 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.566104 14525 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:16:23.589828 14685 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.580470 14773 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:23.528283 14795 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.618206 14794 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.627468 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:23.637703 14801 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.144s	user 0.001s	sys 0.001s
I20260709 13:16:23.626215 14773 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.637545 14435 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.616253 14695 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.649475 14264 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.650815 14360 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.654089 14719 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.609280 14616 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.657353 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.658226 12118 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:23.661486 14616 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.641189 14794 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.656920 14796 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.658438 14435 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:23.679685 14602 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.701478 14787 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.851820 14601 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:23.852775 14721 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.656772 14651 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.853688 14721 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.684405 14701 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.854312 14787 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:23.854897 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.855471 14701 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.856410 14701 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.784348 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1785) took 1177 ms. Trace:
I20260709 13:16:23.799351 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1780) took 1204 ms. Trace:
I20260709 13:16:23.859527 11474 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:23.859404 11401 rpcz_store.cc:276] 0709 13:16:22.594864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.771063 (+176199us) service_pool.cc:224] Handling call
0709 13:16:23.784608 (+1013545us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540014b4d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:23.799123 (+ 14515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372503424}
I20260709 13:16:23.680843 14729 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.660493 14715 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:23.686128 14741 consensus_meta.cc:287] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.603s	user 0.002s	sys 0.012s
W20260709 13:16:23.681466 14768 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.363s	user 0.007s	sys 0.000s
I20260709 13:16:23.858299 11396 rpcz_store.cc:276] 0709 13:16:22.606909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.075788 (+468879us) service_pool.cc:224] Handling call
0709 13:16:23.076174 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.091054 (+ 14880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.783683 (+692629us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b540030be54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.784135 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24723712}
I20260709 13:16:23.864276 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.865073 11546 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:23.680272 14790 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.688791 14774 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.672823 14794 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.677408 14784 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.941948 14794 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.679397 14795 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:23.786456 11472 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1364) took 1462 ms (client timeout 1660 ms). Trace:
I20260709 13:16:23.942844 11551 raft_consensus.cc:2368] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:23.827739 14522 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:23.943071 11472 rpcz_store.cc:269] 0709 13:16:22.323994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.341514 (+ 17520us) service_pool.cc:224] Handling call
0709 13:16:23.785793 (+1444279us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400063454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:23.786133 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789369088}
I20260709 13:16:23.813364 14262 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.805271 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.942009 14784 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.944677 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.945443 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.946206 11477 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:23.828758 14586 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:23.947151 14586 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.948314 11400 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:23.948483 14586 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.615468 14807 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.953747 14807 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.610440 14813 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.818120 14488 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:23.955248 14488 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.956317 14488 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.839641 14685 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.792438 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 201896us
I20260709 13:16:23.842732 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.816958 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2047) took 1330 ms. Trace:
I20260709 13:16:23.799136 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2044) took 1340 ms. Trace:
I20260709 13:16:23.843369 14615 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:23.788846 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2048) took 1301 ms. Trace:
I20260709 13:16:23.800966 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.693107 14736 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:23.845479 14630 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:23.847498 14583 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:23.967733 14488 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.967401 14685 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.968302 14583 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:23.968840 14583 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:23.654874 14648 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.959177 11468 rpcz_store.cc:276] 0709 13:16:22.486503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.960495 (+473992us) service_pool.cc:224] Handling call
0709 13:16:22.960828 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.105491 (+144663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.105521 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.105557 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:23.105588 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.816386 (+710798us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540005a854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.816451 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.816719 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332218112}
I20260709 13:16:23.969952 14583 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.970939 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.971978 14648 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:23.749204 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.972440 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.973229 14615 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:23.956950 14813 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:23.973989 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.862695 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:23.808512 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48200 (request call id 10) took 1766 ms. Trace:
I20260709 13:16:23.976133 11474 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.976300 11397 rpcz_store.cc:276] 0709 13:16:22.042408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.042563 (+   155us) service_pool.cc:224] Handling call
0709 13:16:23.807426 (+1764863us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b54001bc220. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9896520 00007f8da9913652 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.808223 (+   797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41006464}
I20260709 13:16:23.977305 14608 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:23.865250 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 511) took 1904 ms. Trace:
I20260709 13:16:23.979147 11469 rpcz_store.cc:276] 0709 13:16:21.960925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.966531 (+  5606us) service_pool.cc:224] Handling call
0709 13:16:21.978252 (+ 11721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.028800 (+ 50548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.028833 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.029360 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:16:22.031488 (+  2128us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:22.031540 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.031618 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.680805 (+1649187us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:23.864679 (+183874us) raft_consensus.cc:1685] finished
0709 13:16:23.864722 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.865068 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75,"raft.queue_time_us":86928,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":14568,"spinlock_wait_cycles":39183616,"thread_start_us":51954,"threads_started":2,"wal-append.queue_time_us":282766}
I20260709 13:16:23.752658 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1778) took 1158 ms. Trace:
I20260709 13:16:23.982105 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.981137 11398 rpcz_store.cc:276] 0709 13:16:22.594007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.058929 (+464922us) service_pool.cc:224] Handling call
0709 13:16:23.059371 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:23.114798 (+ 55427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.114832 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:23.118416 (+  3584us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.298914 (+180498us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:23.298949 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.360090 (+ 61141us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b54001275d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.360158 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.751681 (+391523us) raft_consensus.cc:1685] finished
0709 13:16:23.751761 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.752351 (+   590us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48866,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":126711552,"thread_start_us":166998,"threads_started":1,"wal-append.queue_time_us":270667}
I20260709 13:16:23.793905 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.784590 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2052) took 1296 ms. Trace:
I20260709 13:16:23.816045 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2040) took 1358 ms. Trace:
I20260709 13:16:23.792459 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:23.904016 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.824942 11515 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:23.985095 11397 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:16:23.698451 14805 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.364s	user 0.001s	sys 0.001s
W20260709 13:16:23.696422 14802 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.355s	user 0.001s	sys 0.001s
W20260709 13:16:23.710565 14804 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 13:16:23.949848 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1412) took 1339 ms. Trace:
I20260709 13:16:23.945497 11472 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:23.963213 11547 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:23.963951 14522 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:24.005077 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.969348 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:23.983843 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.970490 14685 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:23.750819 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 185882us
I20260709 13:16:23.959416 11476 rpcz_store.cc:276] 0709 13:16:22.458806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.848893 (+390087us) service_pool.cc:224] Handling call
0709 13:16:22.849327 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.124606 (+275279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.124634 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.124689 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:23.124720 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.798618 (+673898us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540039a3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.798694 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.798948 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632667904}
I20260709 13:16:23.967458 11549 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
W20260709 13:16:23.966846 11394 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.014s
I20260709 13:16:23.961338 11470 rpcz_store.cc:276] 0709 13:16:22.486810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.992896 (+506086us) service_pool.cc:224] Handling call
0709 13:16:23.008547 (+ 15651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.087445 (+ 78898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.087476 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.087516 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:23.087550 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.788175 (+700625us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b5400214654. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.788275 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.788569 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180964608}
I20260709 13:16:24.001431 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:23.983911 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:23.983484 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.000173 11400 rpcz_store.cc:276] 0709 13:16:22.610316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.959497 (+349181us) service_pool.cc:224] Handling call
0709 13:16:23.949622 (+990125us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400125cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:23.949789 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269156736}
I20260709 13:16:24.008493 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:23.998245 11471 rpcz_store.cc:276] 0709 13:16:22.488230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.264197 (+775967us) service_pool.cc:224] Handling call
0709 13:16:23.264470 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.335015 (+ 70545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.784025 (+449010us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5400213754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.784328 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45589760}
I20260709 13:16:23.999297 11475 rpcz_store.cc:276] 0709 13:16:22.457588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.457855 (+   267us) service_pool.cc:224] Handling call
0709 13:16:22.511334 (+ 53479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.076765 (+565431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.076810 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.076871 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:23.076915 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.815541 (+738626us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400244854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:23.815611 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.815817 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299818368}
I20260709 13:16:24.017324 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:24.018034 11544 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:24.018291 11472 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:24.019635 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:24.020363 11472 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
W20260709 13:16:24.021499 14455 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.008s
I20260709 13:16:24.021692 11472 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:24.022414 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2055) took 1521 ms. Trace:
I20260709 13:16:24.023197 12112 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:24.027792 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.028885 11398 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:24.029652 11396 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:23.976109 14615 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.022745 11472 rpcz_store.cc:276] 0709 13:16:22.501079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.021041 (+1519962us) service_pool.cc:224] Handling call
0709 13:16:24.021447 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.022170 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.022348 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:24.101935 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85979.2us
I20260709 13:16:24.119174 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:24.124221 14832 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:16:24.124626 14239 consensus_meta.cc:287] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.040s	user 0.013s	sys 0.005s
I20260709 13:16:24.126372 14721 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.124650 14813 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:24.106235 11474 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:24.030056 11546 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:24.034405 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.132000 11468 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:24.035357 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:24.103397 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1842) took 1036 ms. Trace:
I20260709 13:16:23.982833 14739 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:24.126507 14799 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:24.069267 11394 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
W20260709 13:16:24.190821 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 155916us
W20260709 13:16:24.191437 14695 consensus_meta.cc:287] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.004s
I20260709 13:16:24.191828 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 523) took 1016 ms. Trace:
I20260709 13:16:24.192060 14773 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.214565 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 524) took 1038 ms. Trace:
I20260709 13:16:24.214902 11547 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:24.219172 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 113902us
W20260709 13:16:24.220057 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 395528us
I20260709 13:16:24.095489 11550 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:24.226926 14608 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.228157 14742 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:24.229053 14737 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.229743 14744 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:24.231974 12085 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 3.278s (SENT)
I20260709 13:16:24.117033 14583 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.239969 11474 rpcz_store.cc:276] 0709 13:16:23.176229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.213475 (+1037246us) service_pool.cc:224] Handling call
0709 13:16:24.213878 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.214134 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.214155 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:24.214215 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:24.214305 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.214367 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.214480 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:24.235811 11545 rpcz_store.cc:276] 0709 13:16:23.091429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.091772 (+   343us) service_pool.cc:224] Handling call
0709 13:16:24.026082 (+934310us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b54002062d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:24.103175 (+ 77093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137575168}
I20260709 13:16:24.239533 11472 rpcz_store.cc:276] 0709 13:16:23.175888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.129564 (+953676us) service_pool.cc:224] Handling call
0709 13:16:24.191133 (+ 61569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.191394 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.191419 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:24.191486 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:24.191529 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.191601 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.191726 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:24.222929 14771 consensus_meta.cc:287] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.003s
I20260709 13:16:24.037601 14790 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.247193 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1414) took 1052 ms. Trace:
I20260709 13:16:24.248409 11394 rpcz_store.cc:276] 0709 13:16:23.197071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.335804 (+138733us) service_pool.cc:224] Handling call
0709 13:16:24.242194 (+906390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:24.267278 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1847) took 1006 ms. Trace:
I20260709 13:16:24.267807 11548 rpcz_store.cc:276] 0709 13:16:23.260986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.263088 (+  2102us) service_pool.cc:224] Handling call
0709 13:16:23.263412 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:23.282222 (+ 18810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.282266 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:23.289998 (+  7732us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.825012 (+535014us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b180019c360. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:24.126007 (+300995us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:24.126078 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.126181 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.224343 (+ 98162us) raft_consensus.cc:1685] finished
0709 13:16:24.224366 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.267092 (+ 42726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18805,"raft.queue_time_us":34310,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8770,"spinlock_wait_cycles":2531712,"thread_start_us":136844,"threads_started":1,"wal-append.queue_time_us":763467}
I20260709 13:16:24.269079 14773 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.246255 14695 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.272378 14695 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.213116 14239 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.274848 14771 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.030558 14761 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.283389 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:24.279757 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 726582us
I20260709 13:16:24.292443 11471 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:16:24.286464 11475 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:24.290130 14819 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.365s	user 0.010s	sys 0.000s
I20260709 13:16:24.303225 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 528) took 1029 ms. Trace:
W20260709 13:16:24.279476 14798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.042s	sys 0.059s
I20260709 13:16:24.306476 14608 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:24.307030 14798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.043s	sys 0.059s
I20260709 13:16:24.307404 14608 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.305373 11475 rpcz_store.cc:276] 0709 13:16:23.273484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.285843 (+1012359us) service_pool.cc:224] Handling call
0709 13:16:24.286221 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.302883 (+ 16662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.303159 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:24.310421 14608 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.302887 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:24.312861 11474 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:24.285481 14771 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.279241 14773 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:24.250231 14744 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.315785 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.316653 14648 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:24.320068 14648 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:24.320218 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 525) took 1143 ms. Trace:
I20260709 13:16:24.320465 14648 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.320592 11468 rpcz_store.cc:276] 0709 13:16:23.176501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.241343 (+1064842us) service_pool.cc:224] Handling call
0709 13:16:24.304090 (+ 62747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.319876 (+ 15786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.319912 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:24.319970 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:24.320003 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.320055 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.320165 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:24.323458 11542 raft_consensus.cc:2368] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:24.324898 14581 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.325644 14761 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.325516 14581 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.327652 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:24.328114 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.329363 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:24.329171 14773 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.304792 14737 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:24.332402 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54156us
I20260709 13:16:24.332624 11471 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:24.335790 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:24.336598 11546 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:24.337325 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.338094 11548 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:24.338681 14744 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.339090 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.341025 14581 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.341912 11469 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:24.345968 14769 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.797s	user 0.001s	sys 0.012s
I20260709 13:16:24.334877 11471 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:24.335953 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1851) took 1073 ms. Trace:
W20260709 13:16:24.350849 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 75924.5us
I20260709 13:16:24.346433 14769 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.349107 11544 rpcz_store.cc:276] 0709 13:16:23.262203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.266215 (+  4012us) service_pool.cc:224] Handling call
0709 13:16:24.335734 (+1069519us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54000b1b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:24.335883 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634049920}
I20260709 13:16:24.332305 14773 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.360107 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.373394 11394 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:24.332448 14737 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.383854 11475 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:24.389329 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.314316 14742 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.391968 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1856) took 1098 ms. Trace:
I20260709 13:16:24.397034 11549 rpcz_store.cc:276] 0709 13:16:23.293470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.026793 (+733323us) service_pool.cc:224] Handling call
0709 13:16:24.389257 (+362464us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b5c00217f0c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dad283161 00007f8dad424a79 00007f8dae8de9b6 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:24.391854 (+  2597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9180928}
I20260709 13:16:24.401345 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:24.424726 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 146513us
I20260709 13:16:24.429876 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:24.433297 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:24.434330 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 73448.8us
I20260709 13:16:24.389135 14608 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:16:24.438535 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 206580us
I20260709 13:16:24.428295 12119 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:24.464810 11469 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:24.440152 14777 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:24.476593 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.480320 11398 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:24.477290 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:24.438107 14648 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.490970 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:24.486716 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:24.492563 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:24.499750 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.479866 14777 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.500169 11544 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:24.502003 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:24.505146 11402 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
W20260709 13:16:24.505466 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86733.4us
I20260709 13:16:24.506513 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1800) took 1102 ms. Trace:
I20260709 13:16:24.506847 11402 rpcz_store.cc:276] 0709 13:16:23.404034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.412295 (+  8261us) service_pool.cc:224] Handling call
0709 13:16:24.506287 (+1093992us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b540019a854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:24.506426 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131498240}
I20260709 13:16:24.507433 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:24.508309 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.512844 14701 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.499251 14239 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.509230 14777 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:24.519681 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:24.523702 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 79120.8us
I20260709 13:16:24.521291 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.515580 14695 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:24.529449 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.541460 11545 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:24.546414 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 527) took 1362 ms. Trace:
I20260709 13:16:24.546849 11474 rpcz_store.cc:276] 0709 13:16:23.183787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.270724 (+1086937us) service_pool.cc:224] Handling call
0709 13:16:24.271123 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.313420 (+ 42297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.546153 (+232733us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:24.546349 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95568256}
I20260709 13:16:24.548550 11474 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:24.551088 14789 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:24.551530 14789 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.551841 14789 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.555200 14299 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.555852 14299 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.559094 14773 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:24.560673 14769 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.542881 14695 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:24.563063 14695 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.563797 11543 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 8. Ops: []
I20260709 13:16:24.566965 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.567565 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2060) took 1217 ms. Trace:
I20260709 13:16:24.567932 11395 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:24.567972 11475 rpcz_store.cc:276] 0709 13:16:23.349996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.310011 (+960015us) service_pool.cc:224] Handling call
0709 13:16:24.310444 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.384411 (+ 73967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.567305 (+182894us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b5400063454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:24.567482 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168019840}
I20260709 13:16:24.569656 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:24.571086 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 103810us
I20260709 13:16:24.571969 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:24.530189 14685 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:24.574851 11548 pending_rounds.cc:85] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 9
I20260709 13:16:24.580233 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:24.579195 14836 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.002s	sys 0.000s
W20260709 13:16:24.588886 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69280.9us
I20260709 13:16:24.590642 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.591655 11395 raft_consensus.cc:2410] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:16:24.595479 14768 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:16:24.578073 14685 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.596690 14685 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:24.597687 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:24.598228 14522 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:24.599422 14731 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:24.579734 11548 pending_rounds.cc:107] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:16:24.600345 11548 raft_consensus.cc:2889] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:24.601687 14702 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:24.602103 14702 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:24.602165 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 522) took 1924 ms. Trace:
I20260709 13:16:24.602540 14702 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.602640 11476 rpcz_store.cc:276] 0709 13:16:22.678126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.031669 (+1353543us) service_pool.cc:224] Handling call
0709 13:16:24.032110 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:24.032733 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:24.032777 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:24.033496 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 13:16:24.371964 (+338468us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:24.372025 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.372115 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.601972 (+229857us) raft_consensus.cc:1685] finished
0709 13:16:24.601994 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.602107 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34619,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":46182,"thread_start_us":317422,"threads_started":1,"wal-append.queue_time_us":402850}
I20260709 13:16:24.587690 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.607980 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:24.587324 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 526) took 1404 ms. Trace:
I20260709 13:16:24.574005 14586 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:24.613996 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 106663us
I20260709 13:16:24.610421 11477 rpcz_store.cc:276] 0709 13:16:23.183452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.242413 (+1058961us) service_pool.cc:224] Handling call
0709 13:16:24.278166 (+ 35753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.437672 (+159506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.437704 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:24.443751 (+  6047us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:24.443795 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.586814 (+143019us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400388bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:24.586859 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.586977 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344735616}
I20260709 13:16:24.619499 11543 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:24.619866 14522 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:24.620306 14522 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:24.620689 14522 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.622361 14807 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:24.623538 14742 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:24.623734 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.626595 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.628034 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.628753 11397 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:24.630210 14773 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:24.631349 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.588310 14239 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:24.631971 14239 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:24.632411 14239 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:24.620609 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:24.646255 11398 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:24.612537 14835 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.090s	user 0.000s	sys 0.003s
I20260709 13:16:24.586148 14701 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:24.651028 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.655203 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.659940 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:24.656767 11542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1841) took 1609 ms (client timeout 1728 ms). Trace:
I20260709 13:16:24.671687 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.680331 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.695911 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:24.663722 11542 rpcz_store.cc:269] 0709 13:16:23.047865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.048011 (+   146us) service_pool.cc:224] Handling call
0709 13:16:24.648698 (+1600687us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003eadd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:24.656693 (+  7995us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776701312}
W20260709 13:16:24.703051 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 82524.7us
I20260709 13:16:24.706139 12118 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:24.653484 14701 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:24.708177 14778 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.715622 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.716610 11543 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:16:24.725787 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:24.729830 14844 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:16:24.730317 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 73798.7us
I20260709 13:16:24.730558 14257 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.731212 14816 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:24.724082 11543 pending_rounds.cc:85] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 3
I20260709 13:16:24.735656 14257 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.737682 11365 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:24.739229 14813 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:24.740666 14789 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.741119 14685 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.744355 14725 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:24.768878 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 148936us
I20260709 13:16:24.745138 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:24.747151 11394 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.749676 11543 pending_rounds.cc:107] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:24.738494 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:24.770972 11543 raft_consensus.cc:2889] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:24.724943 14701 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.726588 14648 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.782029 14257 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.782380 14685 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.783118 14725 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.784585 14789 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.785262 14725 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.786573 14790 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:24.787223 14790 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.789383 14790 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:24.793500 12113 raft_consensus.cc:2410] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:24.749326 14816 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:24.809785 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 72250.4us
I20260709 13:16:24.811476 14257 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:24.811388 14731 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:24.812284 14257 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:24.813426 14257 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:24.817210 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72167.4us
I20260709 13:16:24.820863 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:24.821748 14583 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:24.822000 14837 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.307s	user 0.000s	sys 0.003s
I20260709 13:16:24.826500 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:24.827319 14583 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:24.827275 14702 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.827919 14583 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:24.828370 14583 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.829670 14702 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:24.831949 14843 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 13:16:24.836496 14583 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.859074 14813 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:24.837755 14816 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:24.852756 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 114336us
I20260709 13:16:24.822006 11394 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:24.769891 14778 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:24.880506 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54110us
I20260709 13:16:24.874271 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:24.887894 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.889289 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:24.896062 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 109109us
I20260709 13:16:24.891661 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:24.879101 14586 raft_consensus.cc:2764] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:16:24.912154 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:24.910094 14702 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:24.917220 14849 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:16:24.919160 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.894773 14488 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.926569 11397 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:24.928360 11398 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:24.928377 14849 consensus_queue.cc:1243] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:24.928990 11396 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:24.932641 11543 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:24.917845 14812 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:24.934621 14437 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:16:24.935582 14849 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:24.935985 14849 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:24.936586 14437 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:24.936925 14849 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.937790 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.938408 14849 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.940322 11546 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:24.947388 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:24.948136 11545 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.948482 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.951453 14739 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:24.952071 14789 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:24.952795 14789 consensus_queue.cc:1243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:24.953171 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.953334 14812 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.953723 14812 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.953822 11401 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:24.954793 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.955725 11398 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:24.957115 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.957746 11400 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:24.959475 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.960067 14803 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:24.960296 11398 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:24.960572 14803 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:24.961527 14803 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:24.964033 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.951500 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.917886 14702 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:24.967051 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.967753 11394 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:24.970784 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:24.971480 11402 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:24.968489 11542 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:24.975971 11395 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.964687 14803 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.981268 11398 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:24.881448 14583 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:24.987303 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:24.988008 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:24.989204 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.001035 14522 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.002965 14739 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:25.003479 14739 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:25.004371 14739 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.008302 14299 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:25.009809 14845 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.410s	user 0.000s	sys 0.003s
I20260709 13:16:25.012965 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.013312 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1862) took 1646 ms. Trace:
I20260709 13:16:25.013758 11550 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:25.015846 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.016600 11547 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:24.995819 14702 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.022825 11543 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:25.027132 11545 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:24.977632 14488 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.034426 11546 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:16:25.013732 11551 rpcz_store.cc:276] 0709 13:16:23.367699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.246770 (+879071us) service_pool.cc:224] Handling call
0709 13:16:24.315529 (+ 68759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:24.341014 (+ 25485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.341044 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:24.341544 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:16:24.508043 (+166499us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:24.508075 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.508134 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.009044 (+500910us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:25.013030 (+  3986us) raft_consensus.cc:1685] finished
0709 13:16:25.013050 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.013256 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1182,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"spinlock_wait_cycles":18221312,"thread_start_us":166135,"threads_started":1,"wal-append.queue_time_us":257453}
I20260709 13:16:25.024564 14849 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:25.039364 14522 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:24.988300 14239 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.035588 14849 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.042845 14849 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:25.047411 14842 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.447s	user 0.006s	sys 0.001s
W20260709 13:16:25.076325 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 397383us
I20260709 13:16:25.067888 14778 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:25.117123 14848 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:16:25.144357 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 412311us
W20260709 13:16:25.146760 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 621405us
W20260709 13:16:25.147800 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 217010us
W20260709 13:16:25.152630 14839 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.453s	user 0.000s	sys 0.003s
I20260709 13:16:25.146806 14778 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.148630 14648 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.144156 14799 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.151583 14522 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:16:25.155455 14685 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.188664 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.189456 11398 raft_consensus.cc:2368] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:25.166721 14773 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:25.199144 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 287062us
I20260709 13:16:25.202450 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.203104 11394 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:16:25.206346 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.166530 14695 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.207291 14487 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:25.207674 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.207501 14695 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.207863 14487 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.208779 14685 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.209647 14695 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.209954 14487 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.210203 14695 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.178668 14778 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.235617 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.202643 14813 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.237058 14813 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.241113 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.215287 14239 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:25.217082 14777 consensus_meta.cc:287] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.008s
I20260709 13:16:25.241907 11469 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.242205 14777 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.243654 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.244268 11476 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:25.217990 14812 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.178349 14648 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.246343 14812 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.247798 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:25.226452 11300 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 215) took 32679 ms (client timeout 15000 ms). Trace:
I20260709 13:16:25.226131 14816 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.227492 14685 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.213922 11297 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.191674 14799 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.229233 14488 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.198271 14257 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:16:25.200201 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.232173 14773 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.233630 14522 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:25.235235 11516 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.235180 14695 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.212529 14849 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:25.251030 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50187.1us
W20260709 13:16:25.252982 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 52941us
W20260709 13:16:25.261031 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 329679us
I20260709 13:16:25.262184 11544 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.262253 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:25.251784 14777 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.255520 14685 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.271821 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.272009 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.265763 11471 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:25.273010 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:25.257872 14522 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:25.275096 14239 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:25.282699 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:25.294929 14583 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.282976 11297 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.297281 11297 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.283917 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.285084 11472 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:25.300199 14812 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:25.285540 14522 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.302455 14522 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.303830 14488 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:25.304217 14488 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:25.305063 14488 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.306303 14488 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:25.307720 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.308133 14778 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.308337 11548 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:25.309543 11545 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:25.310072 11395 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:25.311797 11469 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:25.280803 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1376) took 1756 ms. Trace:
I20260709 13:16:25.256374 14807 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:16:25.299412 11297 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.280108 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.288070 14816 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.334908 11394 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:25.337705 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.338518 11548 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:25.342098 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62102.8us
I20260709 13:16:25.342816 11546 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:25.343132 11401 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:25.313290 11468 rpcz_store.cc:276] 0709 13:16:23.524867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.321640 (+796773us) service_pool.cc:224] Handling call
0709 13:16:24.322045 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:24.323023 (+   978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.323050 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:24.323641 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:16:24.453803 (+130162us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:24.453944 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.454225 (+   281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.046611 (+592386us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:25.272958 (+226347us) raft_consensus.cc:1685] finished
0709 13:16:25.272980 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.280736 (+  7756us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1990,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":134,"spinlock_wait_cycles":30679296,"thread_start_us":129385,"threads_started":1,"wal-append.queue_time_us":276125}
I20260709 13:16:25.286772 14739 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.264469 14851 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:16:25.295563 14583 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.359258 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.345887 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:25.333432 14701 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
I20260709 13:16:25.342923 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.348970 11297 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.361967 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.364231 11550 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:25.366351 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.317131 14239 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:25.374415 11547 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.372588 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.374197 11543 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:25.368356 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.366601 14583 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.361263 14813 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.389015 11546 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:25.316727 14685 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:25.390841 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47980us
I20260709 13:16:25.365621 14487 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.389761 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.331090 14731 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:16:25.380736 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.393332 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:25.394683 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.394770 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.396063 14695 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.396926 14849 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.397724 14239 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.399746 14799 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.361022 14812 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:16:25.401014 12119 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:16:25.402242 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.402545 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.450397 11543 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.403414 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:25.382138 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 133566us
I20260709 13:16:25.363039 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:25.407367 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.408910 14866 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.410096 14852 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:25.373955 14864 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.410732 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.411172 14816 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:25.411499 14790 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:25.412053 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.438632 14860 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:25.412525 14777 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.363987 14648 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.414551 14773 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.420765 11545 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.434077 11546 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.406800 14850 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.372211 14868 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.475476 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.441501 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.439245 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.446825 12117 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:25.466089 14777 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.450397 11548 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:25.453899 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:25.251435 11300 rpcz_store.cc:269] 0709 13:15:52.547527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.548509 (+   982us) service_pool.cc:224] Handling call
0709 13:16:25.226209 (+32677700us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:19.297545 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:19.298136 (+   591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:19.298169 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:16:19.298186 (+    17us) tablet.cc:662] Decoding operations
0709 13:16:19.299034 (+   848us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:19.299109 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:19.299137 (+    28us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:16:19.299803 (+   666us) tablet.cc:701] Row locks acquired
0709 13:16:19.299820 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:16:19.299963 (+   143us) write_op.cc:270] Start()
0709 13:16:19.300075 (+   112us) write_op.cc:276] Timestamp: P: 1783602979299909 usec, L: 0
0709 13:16:19.300095 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:16:19.300798 (+   703us) log.cc:844] Serialized 3176 byte log entry
0709 13:16:19.931033 (+630235us) op_driver.cc:464] REPLICATION: finished
0709 13:16:20.507073 (+576040us) write_op.cc:301] APPLY: starting
0709 13:16:20.517314 (+ 10241us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:20.551026 (+ 33712us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:20.570247 (+ 19221us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:20.649875 (+ 79628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:20.650108 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:16:20.650158 (+    50us) write_op.cc:312] APPLY: finished
0709 13:16:20.650443 (+   285us) log.cc:844] Serialized 65 byte log entry
0709 13:16:21.196261 (+545818us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:21.218823 (+ 22562us) write_op.cc:454] Released schema lock
0709 13:16:21.226851 (+  8028us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:21.276081 (+ 49230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:21.283094 (+  7013us) write_op.cc:454] Released schema lock
0709 13:16:21.726734 (+443640us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
Metrics: {"mutex_wait_us":11084,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":553110,"apply.run_cpu_time_us":24044,"apply.run_wall_time_us":771593,"mutex_wait_us":49778,"num_ops":8,"prepare.queue_time_us":87652,"prepare.run_cpu_time_us":16145,"prepare.run_wall_time_us":90023,"raft.queue_time_us":403084,"raft.run_cpu_time_us":24904,"raft.run_wall_time_us":266220,"replication_time_us":630758,"spinlock_wait_cycles":61425408,"thread_start_us":999725,"threads_started":5,"wal-append.queue_time_us":779414,"wal-append.run_cpu_time_us":66755,"wal-append.run_wall_time_us":2724604}]]}
I20260709 13:16:25.455691 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:25.521052 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.455973 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:25.521971 11398 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:25.370916 14701 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.457682 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.458599 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:25.459859 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:25.460793 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:25.449800 14487 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.441509 14851 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.529820 14851 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.465883 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.470487 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.471330 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.473469 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.542677 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.438938 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.475631 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.477605 11477 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:16:25.466363 11297 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 5, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 5 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.475800 14778 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.485886 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.548895 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.550371 11473 raft_consensus.cc:2410] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:25.486647 14854 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:25.490841 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.553017 11399 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.465473 14799 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:25.495123 14868 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.497403 11393 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:16:25.499168 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 91896.3us
I20260709 13:16:25.453460 14852 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:25.506430 11547 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:25.468967 14816 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:25.563678 14816 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.555019 14488 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:25.564612 14488 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:25.469482 14866 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.565500 14488 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.567155 11468 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:25.569079 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.569664 14868 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.571077 11393 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:25.572983 14852 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.574101 14852 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:25.512167 14850 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:25.465361 14860 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.476661 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.517037 11545 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:25.517203 12112 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:25.569679 11394 raft_consensus.cc:2376] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7. Re-sending same reply.
I20260709 13:16:25.518194 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.431615 14857 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:25.463829 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.524616 11401 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:25.524798 11550 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.521476 11400 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:25.523021 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.526537 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.530730 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.530896 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.465534 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.531220 14870 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:25.533203 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.475970 14790 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:25.587956 14860 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:25.566819 14488 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:25.597494 14790 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.599960 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.600062 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.600695 12118 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.601444 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.477216 14864 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.602766 14863 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:25.589387 12114 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:25.604041 14864 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.604099 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.545694 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:25.547304 11516 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:25.605417 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61034.2us
I20260709 13:16:25.523989 14777 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.551875 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.548718 12123 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:25.547704 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.611793 11398 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8.
I20260709 13:16:25.562664 11549 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:25.578118 11475 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:25.611521 11476 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:25.552749 14854 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:25.592047 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.592677 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.562249 14778 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.619772 14854 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.620924 11396 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:25.621109 14854 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.621739 14778 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.596415 11551 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.623778 11551 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:25.624205 11468 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:25.626061 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:25.607102 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61530.7us
I20260709 13:16:25.596436 11472 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.627753 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.628558 11469 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:25.629377 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.630014 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:25.630476 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.633747 12115 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:25.644737 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2050) took 1063 ms. Trace:
I20260709 13:16:25.645187 11549 rpcz_store.cc:276] 0709 13:16:24.581268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.581406 (+   138us) service_pool.cc:224] Handling call
0709 13:16:24.590741 (+  9335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.614208 (+1023467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.644450 (+ 30242us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b5400269b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:25.644646 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234792320}
I20260709 13:16:25.646975 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.648381 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.648990 11475 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.586753 11546 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:25.610544 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:25.615204 12122 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:25.652177 14870 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:25.652571 14870 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:25.653331 14863 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:25.634716 11469 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:25.653896 11546 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:25.630856 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.614018 14702 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.638384 14487 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.639011 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:25.641052 12112 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:25.634140 12118 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:25.655967 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.672540 11474 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:25.673460 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.674609 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.658032 11547 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:25.659991 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:25.579623 14685 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.670188 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:25.660449 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.677680 11469 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:25.613719 11297 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.661639 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.682420 11399 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:25.663410 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:25.667605 11401 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:25.668823 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 121211us
I20260709 13:16:25.668926 14812 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:25.671989 11548 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:25.657341 11397 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:25.670722 14702 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.705070 11402 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:16:25.705549 11402 pending_rounds.cc:85] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 4
I20260709 13:16:25.705835 11402 pending_rounds.cc:107] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:16:25.706142 11402 raft_consensus.cc:2889] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:25.706413 11475 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:25.662201 14701 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:25.697962 11297 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:25.709364 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82081.6us
I20260709 13:16:25.710934 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:25.626531 14583 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.688441 11545 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 13:16:25.609577 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.653832 14813 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:25.698514 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:25.700987 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:25.729142 11477 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.709030 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:25.717280 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.690773 14867 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:25.722642 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:25.724476 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54354.4us
I20260709 13:16:25.703436 11472 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:25.697079 14648 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.730888 11297 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:25.740424 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110529us
I20260709 13:16:25.741623 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.738969 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.677700 14863 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:16:25.763556 11395 raft_consensus.cc:606] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:25.763988 11395 raft_consensus.cc:614] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:25.699993 14866 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.710527 14702 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:25.744359 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 134878us
I20260709 13:16:25.726267 14778 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.710157 14695 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.749989 14715 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:25.751210 14812 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:25.751904 14648 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.753275 11297 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 5 to 8, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:25.736883 14789 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:25.737483 14867 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.757627 14875 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:25.758155 14803 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:25.758751 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67208.6us
I20260709 13:16:25.770692 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.742551 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.777033 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:25.726440 14813 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.774521 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:25.775313 14701 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:25.791307 14812 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:25.791607 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.792405 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:25.811811 11476 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:25.811964 11472 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:25.763314 14773 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:25.800770 11477 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:25.809523 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67070.6us
I20260709 13:16:25.820205 11297 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 5, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:25.797578 14867 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.802248 14863 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:25.819150 14739 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:25.821996 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:25.823211 11469 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:25.781390 14695 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.834110 11397 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:25.807127 14799 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:25.863298 14799 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:25.864378 14799 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.807317 14813 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:25.841711 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54082.7us
I20260709 13:16:25.842670 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:25.850396 14778 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:25.834971 14739 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.823192 14875 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:25.810998 14701 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:25.852825 14872 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:16:25.854319 14648 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.854322 14816 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.855360 14870 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.783798 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:25.841459 14583 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:25.868830 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47307us
I20260709 13:16:25.820174 14803 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:25.873739 14739 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:25.839115 14695 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.890326 14881 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:25.923923 14881 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.924281 14881 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.890733 14701 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.928270 14860 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.930392 14860 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:25.931213 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 133951us
I20260709 13:16:25.893993 14777 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.932046 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:25.933228 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:25.933933 14868 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:25.934309 14868 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.934451 14866 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:25.934674 14868 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:25.934864 14866 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.935322 14866 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:25.938879 14862 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:25.875630 14851 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:25.835238 11297 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 7, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 7 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:25.900219 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57626us
I20260709 13:16:25.915469 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:25.903169 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:25.911494 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 127928us
W20260709 13:16:25.968351 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52962.8us
I20260709 13:16:25.903288 14876 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:25.916189 14873 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:16:25.909356 14870 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.977759 11297 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:25.912127 14648 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:25.943885 14777 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:25.908681 14816 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:25.923067 14257 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
W20260709 13:16:25.988859 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 214387us
W20260709 13:16:25.991196 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 186919us
W20260709 13:16:25.996933 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 93832.6us
I20260709 13:16:25.997620 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:26.018371 11442 leader_election.cc:400] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:26.018735 11442 leader_election.cc:403] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:25.901243 14867 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.994692 14816 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.939478 14862 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:25.994714 14648 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:26.006307 14778 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:26.037338 14799 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.063400 14778 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:26.063493 14799 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.063659 14648 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.064378 14778 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.065104 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:26.065675 14799 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.037925 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.066094 14887 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:26.066605 11546 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.916047 14803 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.067596 14790 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:26.040107 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:26.077234 11548 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:26.067499 14685 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:26.041842 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.078583 11547 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.045559 14867 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.049734 14868 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.053440 14813 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:26.055091 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.994951 14739 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.008826 14437 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:26.058837 14701 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.095088 14701 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.036798 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:26.038396 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:26.074477 11546 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:26.096585 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61174.1us
I20260709 13:16:26.097295 14876 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:26.041224 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.098569 11550 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.100832 11548 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:26.100771 14866 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.066179 14799 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:26.102082 14862 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:26.102236 14799 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:26.079844 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 75321.3us
I20260709 13:16:26.102958 14866 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.010362 14863 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:16:26.104370 14799 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.103987 14881 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:25.901729 14875 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.086328 14790 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.107486 14881 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.108752 14790 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:26.111001 14868 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.019506 14812 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:26.112390 14812 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.114084 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:26.098410 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:26.120688 14852 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:26.122012 14884 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:16:26.120932 14812 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.131006 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.131646 11472 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.134195 11471 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:26.135345 14876 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:26.135392 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.136328 11476 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.136583 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:26.137028 14812 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.139144 14812 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.139351 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.139998 11470 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.141930 11397 raft_consensus.cc:606] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:26.142333 11397 raft_consensus.cc:614] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:26.145727 11472 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:26.138739 14813 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.153975 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.149660 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.154636 11474 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:16:26.087544 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99353.8us
I20260709 13:16:26.170684 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:26.172782 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.173391 11549 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.173676 11547 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:26.174948 11550 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:26.184967 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:26.183991 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 151405us
I20260709 13:16:26.188035 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.186316 14852 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.167300 14583 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:26.195786 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.214723 11475 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:26.227717 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.235018 11477 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:26.232007 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44049.6us
I20260709 13:16:26.238757 11474 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:26.192395 14789 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:26.240645 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44985.2us
I20260709 13:16:26.156697 14702 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:26.243408 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70920.7us
I20260709 13:16:26.174270 14860 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:16:26.194661 14854 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.188488 14488 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:16:26.262620 11476 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:26.271216 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:26.271819 14789 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:26.272054 11400 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:26.272755 14789 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.273654 14875 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:26.274024 14875 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:26.274083 11400 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:26.187254 14807 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:16:26.242090 14583 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:26.274906 14875 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.275580 11398 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:26.276125 14875 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.276887 11550 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:26.279424 14888 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:16:26.285169 14807 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:26.285562 14807 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:26.286119 14739 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.239720 14790 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:26.196751 14868 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:16:26.263337 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.270493 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:26.300709 11394 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:26.308532 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:26.309706 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:26.312637 11401 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.313695 11477 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:26.317044 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:26.317811 12115 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.315814 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:26.323128 12122 raft_consensus.cc:2376] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11. Re-sending same reply.
I20260709 13:16:26.323755 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:26.325677 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 138573us
I20260709 13:16:26.332131 11397 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:26.334197 12115 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:26.336189 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.336941 11545 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:26.247809 14893 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.336226 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.341287 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:26.342759 11396 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:26.345739 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:26.300341 14891 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.351181 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:26.363152 12085 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8.
I20260709 13:16:26.363505 12085 leader_election.cc:403] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:16:26.276075 14583 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.286432 14807 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.372370 14773 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:16:26.347565 14715 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:16:26.339442 14868 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:26.374576 14851 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8.
I20260709 13:16:26.263985 14876 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:16:26.332454 14790 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:26.369951 11401 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:26.383702 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 120479us
I20260709 13:16:26.350404 14799 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:26.399394 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:26.332796 14695 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.393901 11400 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:26.342342 14886 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:16:26.400085 11396 raft_consensus.cc:606] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:26.408036 11396 raft_consensus.cc:614] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:26.403997 14799 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:26.411120 14799 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.391919 14868 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:26.412029 14868 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.350674 14893 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.410436 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:26.403457 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.441253 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:26.446836 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.443095 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:26.460427 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 89142.5us
I20260709 13:16:26.442955 14812 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:26.474550 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:26.479462 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:26.456938 14437 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:26.482064 11396 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:26.458673 14903 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.447229 14899 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.486430 14876 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:26.492060 11549 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:26.483899 14437 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.509219 14892 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.509771 14892 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.483984 14867 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.515226 14912 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:26.515986 14912 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:26.517095 14912 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.517845 14807 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.497124 14790 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.522863 14852 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:26.523551 14702 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.467986 14880 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.524528 14880 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.527932 14695 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.529011 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:26.530813 14854 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:26.531253 14854 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:26.532104 14854 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.534344 14908 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:26.536656 14908 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:26.497901 14909 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:26.535287 14892 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.548023 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:26.561818 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:26.562719 11401 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 4.
W20260709 13:16:26.549587 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 88511.8us
W20260709 13:16:26.590955 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 118509us
I20260709 13:16:26.594127 14854 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.549371 14880 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.599520 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:26.537256 14816 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.617167 14437 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.598647 14868 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.619257 11551 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:26.623538 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:26.625466 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:26.628698 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:26.610841 14910 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:16:26.611385 14860 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:26.588337 14908 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.640271 14648 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:26.640573 14908 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.643721 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.620635 14816 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:26.647209 14860 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:26.648048 14907 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:16:26.648824 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45770.4us
I20260709 13:16:26.648684 14860 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.650163 14867 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:26.650311 14799 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.651165 14867 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:26.652521 14867 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:26.652631 11393 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:26.652954 14867 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:26.653877 14867 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.655314 14867 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.655180 14892 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:26.656675 14892 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:26.654794 14886 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:26.658324 14886 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:26.659302 14886 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.654500 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.663192 14816 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:26.650579 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.664660 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.671352 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.672322 14868 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:26.674886 14701 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:16:26.696110 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45637.2us
I20260709 13:16:26.697105 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.652693 14799 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.705608 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:26.705631 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.707713 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:26.703377 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.710805 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.719766 11393 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:26.663615 14816 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.704526 14868 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:26.724439 14889 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.624s	user 0.005s	sys 0.000s
W20260709 13:16:26.727494 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.704974 14903 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:26.684031 14887 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:26.747589 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:26.750097 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:26.708726 14685 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:26.753240 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.771744 14777 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:26.774320 14921 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.722002 14864 raft_consensus.cc:2764] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:16:26.783085 11468 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:26.751003 14816 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:26.753311 14868 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:26.787070 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:26.784803 14887 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:26.736366 14881 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:16:26.780586 14807 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:26.792228 14893 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.784531 14921 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.792104 14868 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.783315 14685 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.791352 14892 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:26.807858 14887 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.813038 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:26.814381 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1449) took 1044 ms. Trace:
I20260709 13:16:26.815626 14685 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:26.803385 14864 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:26.820216 14864 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:26.821179 14864 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.818292 11399 rpcz_store.cc:276] 0709 13:16:25.769626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.769781 (+   155us) service_pool.cc:224] Handling call
0709 13:16:25.770137 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.779734 (+  9597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.779768 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.802801 (+ 23033us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.035127 (+232326us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:26.035168 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.035243 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.723812 (+688569us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:26.800732 (+ 76920us) raft_consensus.cc:1685] finished
0709 13:16:26.800752 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.814291 (+ 13539us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":789,"raft.queue_time_us":3360,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":34707328,"thread_start_us":145679,"threads_started":1,"wal-append.queue_time_us":231591}
W20260709 13:16:26.824826 14926 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:16:26.830657 12112 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:16:26.831072 12112 pending_rounds.cc:85] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 7
I20260709 13:16:26.831288 12112 pending_rounds.cc:107] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:16:26.831375 14892 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:26.831534 12112 raft_consensus.cc:2889] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:26.851276 11397 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:26.857638 14715 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:26.858114 14715 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:26.859050 14715 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.860219 14715 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:26.864203 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 153499us
I20260709 13:16:26.832017 14921 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.852362 14868 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:26.895345 14933 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.138s	user 0.000s	sys 0.011s
I20260709 13:16:26.895948 14868 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.896704 11469 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:26.897086 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:26.897760 14816 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.898233 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:26.900184 14867 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:26.900575 14867 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:26.901065 14867 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:26.903406 14881 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:26.883118 14892 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:26.905898 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:16:26.906821 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:26.907375 11544 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.908269 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:26.908582 11546 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:26.908986 11549 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:26.911255 11550 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:26.911634 11547 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:26.913482 14695 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.916129 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:26.916476 14923 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.916250 14695 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.917004 14923 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:26.919037 14257 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:26.928167 14257 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:26.930354 14257 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:26.918833 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:16:26.932557 14898 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.532s	user 0.002s	sys 0.000s
I20260709 13:16:26.918721 14695 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.935981 12112 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:16:26.936403 12112 pending_rounds.cc:85] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 8
I20260709 13:16:26.936602 12112 pending_rounds.cc:107] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:16:26.936817 12112 raft_consensus.cc:2889] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:26.946689 11544 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:26.993549 14816 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:26.905148 14239 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:16:26.997716 14867 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.000535 14685 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.003615 14867 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:27.008785 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 72468.7us
W20260709 13:16:27.008224 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 258225us
I20260709 13:16:27.012027 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:27.037983 11544 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 13:16:27.039374 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.040448 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:27.040899 11401 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:27.040027 11550 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:27.042431 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:27.042961 11394 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:27.047358 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:27.052881 11401 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:27.054903 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:27.055953 11401 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:27.073969 14858 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:27.010250 14695 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:16:27.080744 14257 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.081738 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 69805us
I20260709 13:16:27.082321 14899 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:27.085359 14875 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.044196 14923 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:27.087941 14864 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.075779 14877 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:27.074604 14904 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:27.097695 14487 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:27.121982 14875 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.125186 14904 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.125679 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.127228 14904 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.127755 14875 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.122036 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.129074 11474 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:27.130563 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:27.127593 11469 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:27.114897 11394 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:27.115813 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2079) took 1082 ms. Trace:
I20260709 13:16:27.131592 11551 rpcz_store.cc:276] 0709 13:16:26.033388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.033487 (+    99us) service_pool.cc:224] Handling call
0709 13:16:26.040850 (+  7363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.619681 (+578831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.115569 (+495888us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540041dcd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:27.115751 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329800448}
W20260709 13:16:27.118016 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 59326.4us
I20260709 13:16:27.056380 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:27.132826 14739 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:16:27.143031 14899 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:27.143980 14899 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.146131 14487 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:27.146613 14488 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:27.149228 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:27.151542 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 95242.3us
I20260709 13:16:27.157261 11394 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:27.153015 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:27.163976 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:27.164687 12118 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:27.164942 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:27.165982 12113 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:27.165853 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:27.167064 14875 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:27.167448 14875 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.167801 14875 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:27.169623 14799 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.120832 14858 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.171343 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.123426 14877 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:27.175266 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 120202us
I20260709 13:16:27.121001 14257 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.179174 14862 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:27.146129 14912 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.148204 14695 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:27.147720 14488 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:16:27.187672 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.163193 14790 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.183207 14858 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:27.146914 14487 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:27.195390 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:27.176149 14914 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.123052 14864 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.173739 14923 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:27.123436 14777 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:16:27.182847 14860 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:27.209235 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:27.205988 14868 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:16:27.211217 14877 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.197944 14912 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.180941 14876 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:16:27.215152 14914 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.195423 14862 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.219254 14863 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:27.219883 14937 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:27.223100 14923 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.198194 14488 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:27.200567 14790 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:27.198846 14874 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:16:27.225296 14437 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:27.211294 14924 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.436s	user 0.004s	sys 0.000s
I20260709 13:16:27.194478 14695 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:27.173436 14799 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.227368 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:27.266171 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 77349.5us
I20260709 13:16:27.229813 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:27.226922 14257 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:27.255030 14857 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:16:27.284901 14949 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:16:27.294028 14863 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:27.296794 12117 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:27.270582 14923 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:27.233054 14887 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:27.278661 14790 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:27.228333 14868 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:27.228403 14864 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:27.329192 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 101872us
I20260709 13:16:27.354051 11545 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:27.355679 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:27.335774 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:27.297282 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 102505us
I20260709 13:16:27.266340 14812 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:16:27.368346 11442 leader_election.cc:400] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
I20260709 13:16:27.369627 11442 leader_election.cc:403] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:16:27.371728 11547 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:27.347880 14937 raft_consensus.cc:2764] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:16:27.331928 14881 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:27.369864 14813 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.401765 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.405112 11543 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:27.370800 14864 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:16:27.406618 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 207422us
I20260709 13:16:27.403339 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:27.425545 11545 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:27.367067 14887 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.407198 14257 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.458500 14488 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.424520 14937 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:27.409327 14813 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.426501 14881 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:27.675758 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 307599us
I20260709 13:16:27.378624 14487 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.446825 14863 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:27.702977 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 295501us
I20260709 13:16:27.703663 12117 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:27.717916 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:27.727157 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 320588us
W20260709 13:16:27.723412 14951 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.320s	user 0.007s	sys 0.012s
I20260709 13:16:27.743362 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1463) took 1225 ms. Trace:
I20260709 13:16:27.717200 14937 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.744119 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:27.752056 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.710202 14868 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8.
W20260709 13:16:27.744227 14950 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.449s	user 0.004s	sys 0.010s
I20260709 13:16:27.774624 14799 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.767809 14914 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:27.766711 11402 rpcz_store.cc:276] 0709 13:16:26.513301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.513438 (+   137us) service_pool.cc:224] Handling call
0709 13:16:26.513796 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.532065 (+ 18269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.532099 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.568728 (+ 36629us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.680638 (+111910us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:26.680694 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.680781 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.163395 (+482614us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:27.408169 (+244774us) raft_consensus.cc:1685] finished
0709 13:16:27.408201 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.738740 (+330539us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17845,"raft.queue_time_us":18242,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":7503232,"thread_start_us":86816,"threads_started":1,"wal-append.queue_time_us":150007}
I20260709 13:16:27.746081 14963 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:27.804548 12117 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:16:27.813334 14875 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.008s
I20260709 13:16:27.813793 14875 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.738489 14257 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.748798 14923 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.736153 14860 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:16:27.786584 14937 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:27.768417 14790 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.810449 14488 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:27.824493 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 80452.3us
I20260709 13:16:27.826655 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:27.828922 12117 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:27.831136 14958 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:16:27.831445 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:27.832067 14964 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:27.832541 14960 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:27.832651 14964 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.836347 14923 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:27.836757 14960 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.837225 14960 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:27.837692 14813 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.807720 14963 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:27.824360 14912 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.845908 11544 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:27.840152 14860 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:27.907248 11367 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:27.839497 14960 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:27.904909 14858 consensus_meta.cc:287] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.007s
I20260709 13:16:27.838968 14887 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.873704 14864 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:27.913992 11543 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:27.925668 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.926360 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.844971 14874 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:16:27.932165 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.909452 14860 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.980039 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:27.834882 14964 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:27.912278 14812 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:27.929387 14953 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
I20260709 13:16:27.983955 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.913523 14963 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:27.914683 14701 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:27.983342 11469 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:27.913188 14979 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:27.991012 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:27.997247 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.999413 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.000017 11470 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:27.997453 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:28.000759 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.001084 12117 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:27.998037 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.001943 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.997831 11469 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:28.003470 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:27.981130 14881 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.004398 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:28.016360 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.017936 11475 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:28.018095 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.018519 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:27.987797 14867 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.915892 14858 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.022075 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.016572 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:27.982635 14969 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:27.985845 14812 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.020877 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:27.991631 14875 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.039700 11473 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:28.023303 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:28.024477 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:16:28.041178 11476 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:28.041200 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:28.042304 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.042934 11476 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:27.986406 14860 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:27.973129 14864 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.047992 14972 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:28.024101 14978 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:28.004434 14701 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.002595 14964 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:16:28.049799 14790 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:28.067655 14973 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:28.055052 14867 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.021239 11469 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.068892 11469 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.069070 11474 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:28.070422 14867 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.059628 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:27.996068 14970 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:28.071826 14970 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.066977 14978 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:28.072180 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.039726 14789 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:16:28.073467 14978 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.073957 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.011648 14979 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.067101 14974 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:28.066864 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.057180 14858 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.074973 11470 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:16:28.078015 11471 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:28.004261 14976 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:28.041019 14812 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.062685 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:28.063810 14969 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:28.083809 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.085538 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:28.088203 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 63653.6us
I20260709 13:16:28.088585 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.088640 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.092200 11471 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.024812 14773 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:16:28.093621 11470 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:28.092770 14976 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.095324 14488 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:28.100569 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.088407 14964 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:28.088706 14973 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.097267 14488 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.104331 11469 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:28.103603 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.107968 11551 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:28.109097 14970 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.109419 14583 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:28.110057 14583 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:28.110479 14969 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.112277 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.113364 11543 raft_consensus.cc:2368] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:28.116496 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.118065 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:28.119095 14947 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.877s	user 0.000s	sys 0.003s
I20260709 13:16:28.114462 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.118784 11545 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:28.130563 11469 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:28.130883 11477 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:28.138537 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.139721 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.099653 14976 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.142458 14987 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:28.147380 11469 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:28.147547 11545 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:28.148444 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.128600 14912 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:28.148960 11545 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.148896 11469 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:28.158639 11543 raft_consensus.cc:2393] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:28.160727 14488 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.162202 11469 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:28.163133 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:28.165012 14864 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.166253 14957 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:16:28.169530 14986 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:28.171301 11471 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:28.172466 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.151254 14987 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.145912 14701 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:28.186457 12117 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:28.182308 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.180557 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.187074 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.209479 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:28.187526 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.210523 11469 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:28.211589 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.209995 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.218101 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.222409 11545 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:28.174962 11471 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:28.223667 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.224781 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.184708 14648 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.143198 14583 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.227273 14979 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:28.227591 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.156988 14912 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.231796 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.234792 11472 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:28.181324 14987 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.326745 14912 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.250480 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.327701 14583 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.327737 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.328742 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.329671 11471 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:28.235725 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.331214 14989 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:28.331292 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.181017 14937 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.331851 14989 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:28.314728 14976 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.332605 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:28.333038 11468 raft_consensus.cc:2368] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11.
I20260709 13:16:28.332751 14989 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.333626 14937 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.333922 11394 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:28.324195 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.324580 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:28.335867 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.442286 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:28.443970 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.176441 14702 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:28.489288 11545 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.489596 11551 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:28.285861 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.354269 11471 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:16:28.492977 14863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.024s	sys 0.004s
I20260709 13:16:28.335597 11543 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.346117 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:28.323784 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.207808 14488 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:28.495330 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 149315us
I20260709 13:16:28.496488 11399 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:28.349793 11472 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:28.497316 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:28.499101 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.493522 14863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.024s	sys 0.004s
W20260709 13:16:28.500072 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 344325us
I20260709 13:16:28.501953 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.322407 14985 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.505445 11472 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:28.505894 11545 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:28.506810 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.322867 14908 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:28.494257 14992 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.143s	user 0.004s	sys 0.003s
I20260709 13:16:28.351536 14937 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:28.511046 11394 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.507692 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.181689 14860 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.494663 14978 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.490193 14702 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:28.528649 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.529439 11393 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.530143 11545 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:28.524471 11550 raft_consensus.cc:1217] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Deduplicated request from leader. Original: 8.4->[9.5-9.5]   Dedup: 9.5->[]
I20260709 13:16:28.531026 11550 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:16:28.527683 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:28.528044 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.497318 14970 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.532680 11545 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 13:16:28.501911 14877 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:28.533718 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.230517 14812 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:28.534662 11514 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.629s (SENT)
I20260709 13:16:28.546072 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:28.535396 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.546092 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.537757 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.539757 11550 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:16:28.548969 11402 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:28.551728 11550 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:28.551748 11393 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:16:28.505406 14987 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:16:28.554817 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:28.504117 14488 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.551127 14937 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.562453 14488 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.562541 14937 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.563136 14488 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:28.564877 14488 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:28.565353 14488 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:28.566282 14488 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.676345 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:28.554162 14487 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:28.501102 14648 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.567929 15000 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:28.672030 11393 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:16:28.541805 14970 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.672588 11397 raft_consensus.cc:2368] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:28.554452 14773 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.673705 11394 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:28.515076 14985 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.526363 14908 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.332772 14976 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.534384 14978 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.723901 15002 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.724646 15001 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:28.759856 15001 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:28.726887 11514 leader_election.cc:336] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.717s (SENT)
I20260709 13:16:28.764961 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:28.739120 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.739941 11400 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:28.550392 14877 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.742689 11396 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:28.671855 14937 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:28.721218 14867 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:28.756062 15002 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.670861 14988 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:16:28.587422 14702 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.783475 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:28.776337 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 220443us
I20260709 13:16:28.785714 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:28.788228 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61361.4us
I20260709 13:16:28.788661 11399 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:28.769649 14954 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:28.732124 14487 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.804167 11472 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:28.807147 11471 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:28.808588 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.810577 11477 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:28.811096 11477 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:28.816321 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.820209 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.821875 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:28.827863 14987 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:28.828174 14648 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:28.829924 14996 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:28.832448 14970 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:28.836143 14942 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.625s	user 0.006s	sys 0.001s
I20260709 13:16:28.848384 15002 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.849594 15012 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:28.787288 15005 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.850016 15002 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.850255 15012 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.850361 15005 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.852715 15005 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:28.852943 14982 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.800s	user 0.003s	sys 0.010s
I20260709 13:16:28.853251 15005 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:28.853703 15005 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.855281 14976 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.856042 14715 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:28.856587 15011 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:28.856637 14715 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.857241 15011 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.823504 15004 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:28.859520 15004 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.859685 15001 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:28.787967 14867 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.861161 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.861948 11470 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:28.849680 11550 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.863011 14985 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.824329 11477 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.864126 11477 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.777652 14937 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.864898 14695 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:28.865938 14695 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.867333 14937 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.868830 14257 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.852950 15012 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.870079 14877 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.850502 15002 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:28.859875 15004 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:28.848558 11544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2102) took 1708 ms (client timeout 1988 ms). Trace:
W20260709 13:16:28.873579 11544 rpcz_store.cc:269] 0709 13:16:27.140090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.140240 (+   150us) service_pool.cc:224] Handling call
0709 13:16:28.848265 (+1708025us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54001a86d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:28.848480 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540004736}
I20260709 13:16:28.799278 14863 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:28.881775 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1471) took 2193 ms. Trace:
I20260709 13:16:28.884493 11472 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:16:28.894575 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:28.882381 11398 rpcz_store.cc:276] 0709 13:16:26.678686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.679300 (+   614us) service_pool.cc:224] Handling call
0709 13:16:26.701795 (+ 22495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.709350 (+  7555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.709404 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.855256 (+145852us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:26.865503 (+ 10247us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.915639 (+ 50136us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:26.915675 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.921439 (+  5764us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.835323 (+1913884us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:28.871459 (+ 36136us) raft_consensus.cc:1685] finished
0709 13:16:28.871487 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.871654 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150119,"raft.queue_time_us":37611,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9069,"spinlock_wait_cycles":32530560,"thread_start_us":39891,"threads_started":1,"wal-append.queue_time_us":345331}
I20260709 13:16:28.910310 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:28.911342 11399 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:28.925133 11477 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:16:28.925580 11477 pending_rounds.cc:85] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 3
I20260709 13:16:28.925822 11477 pending_rounds.cc:107] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:16:28.926355 11477 raft_consensus.cc:2889] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:28.925671 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:28.858879 14715 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.824820 14812 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.940649 11470 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:28.995338 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:28.999675 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.004431 11401 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:16:28.996385 11472 raft_consensus.cc:2410] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:16:29.011089 11543 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 13:16:29.011708 11543 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:28.972786 15002 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:28.968773 15005 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:29.003549 14583 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.006s
I20260709 13:16:28.975071 14908 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.974268 14487 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:28.964768 15011 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:28.964752 14867 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.062314 11399 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:29.063320 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.072940 11472 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:28.997128 14685 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.062850 14803 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.076083 14952 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.864s	user 0.002s	sys 0.001s
I20260709 13:16:29.079382 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.079725 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2110) took 1718 ms. Trace:
I20260709 13:16:29.080152 11546 rpcz_store.cc:276] 0709 13:16:27.361155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.361495 (+   340us) service_pool.cc:224] Handling call
0709 13:16:27.361912 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.729129 (+367217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.729157 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.729208 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:27.729236 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.079502 (+1350266us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54001ec8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:29.079546 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.079672 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844013440}
I20260709 13:16:29.081533 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.064553 14877 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:29.085919 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:29.091871 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 271943us
I20260709 13:16:29.093863 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.099931 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:29.105881 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:29.107604 14917 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.109907 15014 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:29.111510 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.112146 11394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:29.112793 11442 leader_election.cc:400] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:29.112735 11394 raft_consensus.cc:606] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.113245 11394 raft_consensus.cc:614] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:29.113296 11395 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:29.115258 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.115914 11442 leader_election.cc:403] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:29.116639 14908 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:29.118141 14908 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:29.092491 14891 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:16:29.119500 14891 consensus_queue.cc:1243] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:16:29.120047 14891 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:29.120432 14891 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:29.101245 14877 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.125589 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.128484 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:29.129096 11396 raft_consensus.cc:606] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.121320 14891 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.132776 14877 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.133435 15014 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.088968 14803 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.133960 15014 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.130715 14891 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:29.127626 11401 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:29.141068 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.141669 11401 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:29.142444 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.136698 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.111828 14685 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:29.149996 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.151599 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.153290 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.121932 14977 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:29.158944 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.156675 11399 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:29.157521 11401 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:29.156792 12122 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:29.162366 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.160597 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:29.163022 11396 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.157110 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.165832 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.166868 14893 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.166813 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.167980 11473 raft_consensus.cc:2410] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:16:29.149317 14867 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:29.170166 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:16:29.171165 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.172219 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.173170 11396 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:29.173287 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.163012 11401 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:29.174005 11476 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:29.174561 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.174930 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.176527 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.176790 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.177563 11477 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:29.178421 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.179160 11476 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:29.179594 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.179726 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.157774 14979 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:16:29.180330 11472 raft_consensus.cc:2376] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:29.177161 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:29.180704 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.180106 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.181644 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.182276 11473 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:29.183130 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.183321 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:29.186743 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.187573 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.179957 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.190025 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.191272 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.192235 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.193732 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.193830 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.194931 11470 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:29.195816 11473 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:29.196657 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.197995 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.190572 11472 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:29.202517 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:29.185665 15017 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:16:29.199672 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.234448 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.168565 14977 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:29.231841 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.157419 14886 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:16:29.170584 14685 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:29.240828 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:29.233681 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:29.154667 14863 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:16:29.154815 14812 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:16:29.158707 14887 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:16:29.234902 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.215984 14803 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.238162 14851 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:29.236099 11473 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:29.239145 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.135684 15004 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.260937 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.194619 14877 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:29.240950 11365 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:29.216136 14899 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.216548 15011 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:29.147554 14908 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.160928 14813 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:16:29.233737 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.248992 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.232309 14966 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:29.186514 14893 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:16:29.254060 14851 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:29.254578 14887 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.255254 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.270429 14976 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:29.256004 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.270015 15020 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:29.257342 12123 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:29.282366 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.257992 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.259750 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.233804 15014 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.271895 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.272643 12117 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:29.272933 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.295653 12116 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
W20260709 13:16:29.274223 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 103118us
I20260709 13:16:29.271194 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.276458 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:29.251223 14969 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:29.278373 11470 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:29.284680 12118 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
W20260709 13:16:29.255789 15006 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.400s	user 0.005s	sys 0.000s
I20260709 13:16:29.287057 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.256313 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:29.273110 14988 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.289335 11393 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:29.289350 12119 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:29.300432 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.301146 12116 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:29.300429 12114 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.296818 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.280339 14857 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:29.296209 11515 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.299695 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.277626 14899 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.406603 15004 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.643952 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:29.644160 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 352135us
I20260709 13:16:29.645108 11398 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:29.645309 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:29.530233 15022 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:16:29.646409 14893 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.646831 14893 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.647238 14893 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.365734 12113 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:16:29.360630 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 119871us
W20260709 13:16:29.594178 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 302003us
W20260709 13:16:29.650036 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.419454 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.285413 14851 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:29.651165 11394 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:29.502954 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1834) took 1142 ms. Trace:
I20260709 13:16:29.651901 11400 rpcz_store.cc:276] 0709 13:16:28.360459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.499487 (+139028us) service_pool.cc:224] Handling call
0709 13:16:28.499765 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.767429 (+267664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.502687 (+735258us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540030aa54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:29.502890 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542863616}
W20260709 13:16:29.654585 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 357817us
I20260709 13:16:29.655362 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.360335 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:29.536952 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.657346 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.498893 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:29.390813 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.422966 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.287202 14976 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:29.390089 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.478520 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.695273 12116 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:29.648569 15027 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:29.699880 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.288568 15020 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:29.421723 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:29.286080 14966 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:29.695271 14739 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.736398 14980 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:16:29.722123 11394 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:29.743366 11546 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:29.303325 14979 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:29.298205 14978 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:29.539884 15030 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:16:29.391721 14977 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.297775 14852 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:29.503103 15024 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:16:29.593580 14917 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.299860 14877 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:29.772173 14877 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.496732 14860 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.406136 15016 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.773573 14979 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.772878 14877 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:29.752997 15029 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.718964 14239 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:16:29.796494 11473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.000s	sys 0.054s
I20260709 13:16:29.700809 14976 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:29.702390 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:29.798957 12116 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:29.648192 14899 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.651297 14857 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:29.701345 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46068.4us
I20260709 13:16:29.801699 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.737108 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.500048 14903 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:29.749702 12114 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
W20260709 13:16:29.506727 14695 consensus_meta.cc:287] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.001s
I20260709 13:16:29.757990 14893 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.754395 14739 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:29.774772 15016 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.360205 14988 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.804677 14903 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:29.811789 11393 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:29.812294 14903 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.813664 14903 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:29.814630 14988 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.815670 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.795238 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.820663 11546 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:29.821867 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.822727 11546 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:29.743549 15020 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:29.797091 11473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.000s	sys 0.054s
I20260709 13:16:29.800410 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:29.746085 14648 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.804015 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.803916 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.360585 14969 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:29.588495 15026 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 13:16:29.825291 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1832) took 1510 ms. Trace:
I20260709 13:16:29.830049 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.831708 11548 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:29.805960 14857 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:29.834626 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.835225 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:29.835747 11546 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:29.836719 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:29.832435 11395 rpcz_store.cc:276] 0709 13:16:28.315232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.315399 (+   167us) service_pool.cc:224] Handling call
0709 13:16:29.825113 (+1509714us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400199954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:29.825225 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703040256}
I20260709 13:16:29.839915 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.810384 14239 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.809340 14739 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.820257 15016 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.798259 15029 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.841789 14799 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:29.841044 15027 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.851470 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.832892 14988 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:29.852649 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.859148 11470 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:29.861940 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:29.856714 11546 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:29.478005 14685 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.833690 14978 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:29.857760 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.843772 14813 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:29.843422 14695 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:29.852089 14867 consensus_meta.cc:287] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.003s
I20260709 13:16:29.858899 14799 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:29.874953 12122 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:29.875049 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.833629 14648 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.861712 14988 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.872587 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.844866 14851 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.876564 15033 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.881438 11441 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.889346 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:29.896008 11470 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:29.898665 11470 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:29.880522 14867 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:29.892426 14988 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.882185 11546 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:29.843863 14966 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.913844 11542 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:29.913897 11475 raft_consensus.cc:2368] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7.
W20260709 13:16:29.915678 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75845.7us
I20260709 13:16:29.916540 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.877735 14978 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.920984 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:29.923118 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.927333 11442 leader_election.cc:400] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:29.927687 11442 leader_election.cc:403] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:29.928256 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:29.917577 14488 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.019s	sys 0.012s
I20260709 13:16:29.929510 11546 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:29.931977 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.933141 11546 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
W20260709 13:16:29.920215 14891 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.000s
W20260709 13:16:29.930897 14488 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.019s	sys 0.013s
I20260709 13:16:29.931058 14799 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.934514 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.941087 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:29.941210 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.914621 15037 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:29.889691 14969 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.943269 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:29.945735 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:29.936069 15036 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.947235 14893 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.942585 14867 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:16:29.952051 14648 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.953570 14851 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.944063 14937 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:16:29.955904 14937 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:29.956269 14937 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:29.957165 14937 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.958460 14937 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:29.945264 14891 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:29.959649 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1426) took 1134 ms. Trace:
I20260709 13:16:29.960021 11469 rpcz_store.cc:276] 0709 13:16:28.825348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.825669 (+   321us) service_pool.cc:224] Handling call
0709 13:16:28.872975 (+ 47306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:28.925243 (+ 52268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.925276 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:28.940789 (+ 15513us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.174592 (+233803us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:29.203755 (+ 29163us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.959220 (+755465us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b540025dda0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9894733 00007f8da9919dbf 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c
0709 13:16:29.959270 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.959345 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.959387 (+    42us) raft_consensus.cc:1685] finished
0709 13:16:29.959471 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.959595 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40832,"prepare.run_cpu_time_us":267,"prepare.run_wall_time_us":594,"raft.queue_time_us":11757,"raft.run_cpu_time_us":2640,"raft.run_wall_time_us":49032,"spinlock_wait_cycles":82852096,"thread_start_us":111828,"threads_started":1,"wal-append.queue_time_us":115364}
I20260709 13:16:29.963654 14877 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:29.966041 14877 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:29.968132 14978 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:29.968523 14968 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 2.217s	user 0.007s	sys 0.003s
W20260709 13:16:29.971475 15035 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:16:29.973796 15038 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:16:29.975056 14852 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:29.974499 14988 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.986071 14988 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:29.986255 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:29.986896 12114 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.990271 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:29.992856 14984 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.942s	user 0.000s	sys 0.006s
I20260709 13:16:29.994107 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 644) took 2159 ms. Trace:
I20260709 13:16:29.994467 12112 rpcz_store.cc:276] 0709 13:16:27.834468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.834614 (+   146us) service_pool.cc:224] Handling call
0709 13:16:27.834939 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.835294 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.835322 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.835746 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.971654 (+135908us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.971693 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.971765 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.992309 (+2020544us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:29.993788 (+  1479us) raft_consensus.cc:1685] finished
0709 13:16:29.993809 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.994030 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1685,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":31697792,"thread_start_us":135514,"threads_started":1,"wal-append.queue_time_us":214485}
I20260709 13:16:29.995590 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.001207 11402 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:29.942857 14977 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7.
I20260709 13:16:30.004803 14893 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:30.005252 14862 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.005429 14893 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:29.943447 14799 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.008342 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.008612 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1905) took 1902 ms. Trace:
I20260709 13:16:30.009142 11550 rpcz_store.cc:276] 0709 13:16:28.106608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.559908 (+453300us) service_pool.cc:224] Handling call
0709 13:16:28.849471 (+289563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.862779 (+ 13308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.008328 (+1145549us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54001a86d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.008534 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665014656}
I20260709 13:16:30.010200 14963 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.010293 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.010751 14963 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:30.011238 14963 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.018839 11546 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:30.013052 11550 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:30.021780 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.022373 11400 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.028146 11469 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:30.018157 12115 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:30.031661 12114 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:30.035197 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.038070 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.039350 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.045053 11395 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.041462 14863 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:30.049113 14863 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.044509 14257 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:16:30.052665 14997 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.498s	user 0.000s	sys 0.002s
I20260709 13:16:30.053915 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.038419 14852 raft_consensus.cc:2764] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:16:30.060658 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.075852 15031 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.251s	user 0.001s	sys 0.001s
W20260709 13:16:30.076388 15032 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:16:30.012221 15033 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:30.041525 14648 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:30.041996 14977 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:30.106184 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 162978us
I20260709 13:16:30.107970 14799 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.108240 14702 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:30.111487 11400 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:30.115535 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.114972 11550 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:30.130852 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.111132 14257 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:30.127336 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:30.134420 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.136535 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.141781 14863 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.141554 14857 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.131052 15029 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.143813 14857 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.123312 14893 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.144217 14739 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.141554 14976 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.146324 14739 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.146450 14976 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:30.148213 14962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.101s	user 0.005s	sys 0.013s
W20260709 13:16:30.148576 14962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.101s	user 0.005s	sys 0.013s
I20260709 13:16:30.130765 14977 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.139253 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.153901 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.156204 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.157624 11550 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:30.158043 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:30.158849 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.161046 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:30.164884 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.175099 15016 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.166597 14257 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.177000 11550 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:30.183094 15016 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.183614 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1851) took 1026 ms. Trace:
I20260709 13:16:30.184052 11393 rpcz_store.cc:276] 0709 13:16:29.158057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.695106 (+537049us) service_pool.cc:224] Handling call
0709 13:16:29.746595 (+ 51489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.812196 (+ 65601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.183374 (+371178us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b5400349754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:30.183548 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149535488}
I20260709 13:16:30.185676 11395 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:30.186669 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:30.187673 11542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2114) took 2808 ms (client timeout 2184 ms). Trace:
W20260709 13:16:30.188048 11542 rpcz_store.cc:269] 0709 13:16:27.378936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.403072 (+ 24136us) service_pool.cc:224] Handling call
0709 13:16:30.187502 (+2784430us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400404554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.187584 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4840816512}
I20260709 13:16:30.189379 11542 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:30.189877 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2132) took 1062 ms. Trace:
I20260709 13:16:30.189961 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.190229 11542 rpcz_store.cc:276] 0709 13:16:29.127557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.188861 (+1061304us) service_pool.cc:224] Handling call
0709 13:16:30.189108 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.189663 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.189834 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:30.146410 15020 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.191045 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.192785 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.193114 15020 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.193364 11397 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:30.194067 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.194300 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.195201 12117 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:30.195667 11401 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:30.195971 15033 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:30.196480 15033 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:30.196707 14648 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:30.197384 15033 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.186796 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2131) took 1064 ms. Trace:
I20260709 13:16:30.200498 14977 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:30.200316 11546 rpcz_store.cc:276] 0709 13:16:29.122655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.186086 (+1063431us) service_pool.cc:224] Handling call
0709 13:16:30.186354 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.186511 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.186536 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.186571 (+    35us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:30.186598 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.186643 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.186746 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:30.201766 11365 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.838s (SENT) [suppressed 30 similar messages]
I20260709 13:16:30.195703 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2128) took 1116 ms. Trace:
I20260709 13:16:30.202934 11395 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:30.202392 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2133) took 1066 ms. Trace:
I20260709 13:16:30.204007 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:30.204660 15045 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:16:30.204732 11393 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:30.205929 14921 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:30.206408 14921 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:30.160144 14959 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:30.207052 14959 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:30.207437 14959 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.207262 14921 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.208312 14921 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.212000 14954 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.212327 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.212442 14954 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.212529 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:30.213003 11395 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:30.213224 11469 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.213846 11473 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:16:30.214236 11473 pending_rounds.cc:85] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 5
I20260709 13:16:30.214488 11473 pending_rounds.cc:107] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:16:30.214732 11473 raft_consensus.cc:2889] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.215142 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2148) took 1033 ms. Trace:
I20260709 13:16:30.215481 11471 rpcz_store.cc:276] 0709 13:16:29.182077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.233954 (+ 51877us) service_pool.cc:224] Handling call
0709 13:16:29.236154 (+  2200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.236467 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.236491 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:29.242952 (+  6461us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.433317 (+190365us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:29.433374 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.433475 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.001008 (+567533us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b48003702d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:30.152746 (+151738us) raft_consensus.cc:1685] finished
0709 13:16:30.152776 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.215080 (+ 62304us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":2621,"raft.run_wall_time_us":763829,"spinlock_wait_cycles":6637952,"thread_start_us":174105,"threads_started":1,"wal-append.queue_time_us":488657}
W20260709 13:16:30.217618 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54689.6us
I20260709 13:16:30.218757 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:30.202790 11548 rpcz_store.cc:276] 0709 13:16:29.078903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.194925 (+1116022us) service_pool.cc:224] Handling call
0709 13:16:30.195217 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.195377 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.195399 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.195436 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:30.195462 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.195538 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.195650 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:30.213891 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.222431 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1434) took 1045 ms. Trace:
I20260709 13:16:30.222790 11475 rpcz_store.cc:276] 0709 13:16:29.172568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.174557 (+  1989us) service_pool.cc:224] Handling call
0709 13:16:30.218238 (+1043681us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400213c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.218363 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654476544}
I20260709 13:16:30.223232 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.224785 14813 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:30.181303 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2129) took 1078 ms. Trace:
W20260709 13:16:30.226495 11468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2124) took 2236 ms (client timeout 1838 ms). Trace:
W20260709 13:16:30.227087 11468 rpcz_store.cc:269] 0709 13:16:27.984152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.984330 (+   178us) service_pool.cc:224] Handling call
0709 13:16:30.220500 (+2236170us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540028bc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.220643 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765524736}
I20260709 13:16:30.228754 11550 rpcz_store.cc:276] 0709 13:16:29.103324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.166138 (+1062814us) service_pool.cc:224] Handling call
0709 13:16:30.176811 (+ 10673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.181045 (+  4234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.181245 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:30.222199 11393 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.229450 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.231285 11542 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:30.217993 14648 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:30.231911 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 73157.9us
I20260709 13:16:30.220913 14966 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:30.244437 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:30.231600 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.204103 11546 rpcz_store.cc:276] 0709 13:16:29.135955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.201532 (+1065577us) service_pool.cc:224] Handling call
0709 13:16:30.201848 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.202056 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.202083 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.202121 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:30.202169 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.202241 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.202340 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:30.246347 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.247195 14978 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:30.247943 14963 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.248644 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.249694 14978 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:30.250191 14963 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.251328 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:30.252542 11550 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:30.253362 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:30.253923 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.254925 11546 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:30.236224 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 295063us
I20260709 13:16:30.240252 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:30.243335 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.243120 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:30.351569 11468 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
W20260709 13:16:30.351933 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 108957us
I20260709 13:16:30.234935 11402 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:30.353483 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:30.231416 11400 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.245115 14876 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:30.240619 14976 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.251611 15029 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:30.342285 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.360260 11542 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:30.230561 15027 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.230335 14970 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.217767 14799 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:16:30.369102 15046 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:16:30.342952 14966 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:30.334524 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77627.1us
I20260709 13:16:30.230964 14977 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:30.230242 14858 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:30.354568 11475 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:30.377985 11475 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:30.354980 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.350449 11393 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:30.355509 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2122) took 2360 ms. Trace:
I20260709 13:16:30.378952 11474 rpcz_store.cc:276] 0709 13:16:27.983228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.983413 (+   185us) service_pool.cc:224] Handling call
0709 13:16:27.983681 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.069470 (+ 85789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.343225 (+2273755us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b540039bcd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:30.343468 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195561728}
W20260709 13:16:30.347720 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 96378.9us
I20260709 13:16:30.347285 11469 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:30.347549 14908 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:30.363786 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:30.343925 15044 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.087s	user 0.000s	sys 0.003s
I20260709 13:16:30.386276 11468 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.373447 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:30.374478 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:30.381278 11469 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:30.389047 12114 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:30.343109 14963 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:30.390272 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:30.393536 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:30.358594 14994 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:30.395843 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:30.356904 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.397733 14858 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.398774 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:30.408241 11393 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:30.355930 14876 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:30.370335 14891 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:30.402158 11395 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:30.406971 11399 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:30.409052 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:30.437822 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:30.438444 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44920.2us
I20260709 13:16:30.397285 14257 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.445837 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:30.403716 14977 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:30.415958 14876 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.373473 15027 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.443126 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:30.461772 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:30.492208 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:30.416285 14891 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:30.417130 14994 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:30.574760 12115 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:30.478776 12114 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:30.543238 14891 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.393900 14648 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:16:30.475543 15028 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:30.502256 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:30.599790 11469 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.597733 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2113) took 3210 ms. Trace:
W20260709 13:16:30.512311 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.599108 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2111) took 3231 ms. Trace:
I20260709 13:16:30.514626 14876 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.446935 15048 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:16:30.516108 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.517159 14937 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:30.518018 12122 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:30.579780 14777 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.520691 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 654) took 1226 ms. Trace:
I20260709 13:16:30.529592 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.532272 14858 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.534196 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1444) took 1352 ms. Trace:
I20260709 13:16:30.615628 11477 rpcz_store.cc:276] 0709 13:16:29.181742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.186538 (+  4796us) service_pool.cc:224] Handling call
0709 13:16:30.533954 (+1347416us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54002a3854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.534120 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2920511872}
W20260709 13:16:30.542886 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.544697 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.500905 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.456207 14257 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.457280 14966 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.620297 11469 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:30.510285 14977 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.634948 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2124) took 1557 ms. Trace:
I20260709 13:16:30.635088 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:30.635708 11399 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:30.635514 11544 rpcz_store.cc:276] 0709 13:16:29.077847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.079997 (+  2150us) service_pool.cc:224] Handling call
0709 13:16:29.080313 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.080535 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:30.399818 (+1319283us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.604021 (+204203us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.634123 (+ 30102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.634187 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:30.634239 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.634767 (+   528us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.634883 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1348913,"prepare.queue_time_us":1373815,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":744,"spinlock_wait_cycles":15760640,"thread_start_us":203508,"threads_started":1}
I20260709 13:16:30.575201 12116 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
W20260709 13:16:30.577263 11367 connection.cc:256] RPC call timeout handler was delayed by -1.2508468219998576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
I20260709 13:16:30.507342 14685 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.602605 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 652) took 1316 ms. Trace:
I20260709 13:16:30.519618 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2160) took 1277 ms. Trace:
I20260709 13:16:30.639286 11470 rpcz_store.cc:276] 0709 13:16:29.241997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.902567 (+660570us) service_pool.cc:224] Handling call
0709 13:16:29.902825 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.938545 (+ 35720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.938578 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:29.938622 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:29.938657 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.519326 (+580669us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b540039add4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:30.519392 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.519550 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81685888}
I20260709 13:16:30.612089 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.579432 14863 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.612955 12117 rpcz_store.cc:276] 0709 13:16:29.294151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.299463 (+  5312us) service_pool.cc:224] Handling call
0709 13:16:30.520450 (+1220987us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400760a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.520573 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233903360}
I20260709 13:16:30.640712 11542 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.574007 11550 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:30.602741 15053 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.068s	user 0.000s	sys 0.003s
W20260709 13:16:30.630415 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 428644us
W20260709 13:16:30.630636 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 101135us
I20260709 13:16:30.632795 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.601841 11549 rpcz_store.cc:276] 0709 13:16:27.363173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.363368 (+   195us) service_pool.cc:224] Handling call
0709 13:16:27.363744 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.364024 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:27.609616 (+245592us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.416465 (+1806849us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b480002d9d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:30.252079 (+835614us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b1800455f80. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8da682ea40 00007f8da68294e4 00007f8da6829455 00007f8dad3113c7 00007f8da99239a7 00007f8da99330bf 00007f8da9933038
0709 13:16:30.252317 (+   238us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.593935 (+341618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.594012 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:30.594078 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.594244 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.594436 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2524236,"prepare.queue_time_us":2977154,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":6696,"raft.queue_time_us":75279,"raft.run_cpu_time_us":8014,"raft.run_wall_time_us":1640424,"spinlock_wait_cycles":16598656,"thread_start_us":679264,"threads_started":3}
I20260709 13:16:30.699329 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:30.700143 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.701649 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:30.674557 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.601106 11547 rpcz_store.cc:276] 0709 13:16:27.377055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.377186 (+   131us) service_pool.cc:224] Handling call
0709 13:16:27.377528 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:27.730364 (+352836us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:27.864203 (+133839us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.996243 (+132040us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:27.996912 (+   669us) log.cc:844] Serialized 52 byte log entry
0709 13:16:27.997065 (+   153us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:28.168890 (+171825us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:30.369841 (+2200951us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.568027 (+198186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.568134 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.586839 (+ 18705us) raft_consensus.cc:1685] finished
0709 13:16:30.586860 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.587020 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4892437,"prepare.queue_time_us":2477921,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":379385,"raft.run_cpu_time_us":480,"raft.run_wall_time_us":29113,"spinlock_wait_cycles":19719424,"thread_start_us":640267,"threads_started":4,"wal-append.queue_time_us":147998,"wal-append.run_cpu_time_us":3627,"wal-append.run_wall_time_us":1124773}
I20260709 13:16:30.704867 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:30.706118 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2126) took 1627 ms. Trace:
I20260709 13:16:30.599434 15028 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.638533 12113 rpcz_store.cc:276] 0709 13:16:29.279938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.296985 (+ 17047us) service_pool.cc:224] Handling call
0709 13:16:29.298581 (+  1596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.649247 (+350666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.596124 (+946877us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5400745250. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:30.596313 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3015552}
I20260709 13:16:30.728878 11542 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:30.706485 11543 rpcz_store.cc:276] 0709 13:16:29.078401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.102432 (+ 24031us) service_pool.cc:224] Handling call
0709 13:16:29.102833 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.103171 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:30.434134 (+1330963us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.705778 (+271644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.705837 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:30.705879 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.705949 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.706074 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1296801,"prepare.queue_time_us":1353513,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":3375,"raft.queue_time_us":570,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":271342,"threads_started":1}
I20260709 13:16:30.596024 14971 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:30.611596 14648 consensus_queue.cc:1243] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:30.740506 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:30.745731 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.038s	sys 0.046s
W20260709 13:16:30.775781 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.039s	sys 0.049s
W20260709 13:16:30.776142 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 549318us
I20260709 13:16:30.738955 11547 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:30.761581 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2149) took 1578 ms. Trace:
W20260709 13:16:30.748972 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 109491us
I20260709 13:16:30.636686 14978 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.777930 11472 rpcz_store.cc:276] 0709 13:16:29.182858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.235313 (+ 52455us) service_pool.cc:224] Handling call
0709 13:16:29.239682 (+  4369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.249106 (+  9424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.249127 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:29.249439 (+   312us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.408206 (+158767us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:29.408254 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.408332 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.503969 (+ 95637us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b48003702d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:30.760602 (+1256633us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b2400296a98. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:16:30.761261 (+   659us) raft_consensus.cc:1685] finished
0709 13:16:30.761296 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.761501 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":1148,"raft.run_wall_time_us":260890,"spinlock_wait_cycles":47318656,"thread_start_us":125534,"threads_started":1,"wal-append.queue_time_us":524095}
I20260709 13:16:30.599831 14959 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.780431 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:30.695290 14685 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.636875 14813 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.694262 15037 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:30.790966 12119 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:30.791975 14959 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.792091 14876 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.792711 14685 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:30.793428 15054 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:30.793821 14257 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.780916 14969 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:30.794509 11468 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:30.732795 14863 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.629819 14777 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.795549 14969 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.796034 14969 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.796147 15076 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.796725 15076 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.797163 14777 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:30.798273 14863 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:30.798847 14702 consensus_meta.cc:287] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.003s
I20260709 13:16:30.782720 14979 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.799764 14979 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.800885 15036 raft_consensus.cc:2764] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:16:30.741367 15028 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.801760 14739 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.739362 15048 consensus_queue.cc:1243] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:16:30.818408 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:30.808799 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.858542 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2152) took 1671 ms. Trace:
I20260709 13:16:30.780570 14648 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:30.845017 11514 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.761s (SENT)
I20260709 13:16:30.862082 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:30.867884 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 49522.9us
I20260709 13:16:30.869115 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2120) took 2896 ms. Trace:
W20260709 13:16:30.863327 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54676.9us
I20260709 13:16:30.873704 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.869015 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:30.883363 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.821897 14695 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.869797 11551 rpcz_store.cc:276] 0709 13:16:27.972486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.498705 (+526219us) service_pool.cc:224] Handling call
0709 13:16:28.510719 (+ 12014us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:28.511249 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.511275 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:28.516335 (+  5060us) log.cc:844] Serialized 52 byte log entry
0709 13:16:28.655166 (+138831us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:30.733360 (+2078194us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.868745 (+135385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.868821 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.868862 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:30.868873 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.869062 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1719736,"prepare.queue_time_us":1743410,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":502,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8889,"thread_start_us":559739,"threads_started":3,"wal-append.queue_time_us":177203,"wal-append.run_cpu_time_us":2022,"wal-append.run_wall_time_us":1022972}
I20260709 13:16:30.781231 14976 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:16:30.807664 14813 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.862573 14648 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:30.900722 12083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.020s
I20260709 13:16:30.867300 14921 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:30.802101 14978 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:30.906095 12083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.012s	sys 0.021s
W20260709 13:16:30.891296 15067 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 13:16:30.893683 14954 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.000s
I20260709 13:16:30.860262 14739 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.887420 14816 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:30.877775 14973 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:30.871560 14862 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:30.894614 14909 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:30.906538 14863 raft_consensus.cc:2764] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:16:30.894165 15061 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:30.859782 11476 rpcz_store.cc:276] 0709 13:16:29.186978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.245100 (+ 58122us) service_pool.cc:224] Handling call
0709 13:16:29.245468 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.284998 (+ 39530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.285027 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:29.288861 (+  3834us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.288995 (+   134us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:29.293526 (+  4531us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.957344 (+663818us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18004c2ee0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:30.693430 (+736086us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b18004c2ee0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8da9911060 00007f8da9923716 00007f8da9934287 00007f8da9934208
0709 13:16:30.858004 (+164574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.858287 (+   283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.858329 (+    42us) raft_consensus.cc:1685] finished
0709 13:16:30.858341 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.858478 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56419,"prepare.queue_time_us":321,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":248334,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":2554,"spinlock_wait_cycles":10156672,"thread_start_us":203443,"threads_started":2}
I20260709 13:16:30.917697 14773 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:16:30.918439 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.922856 11475 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.924383 14891 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.858950 14994 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.914713 14648 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.928889 14702 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:30.933054 11396 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:30.897395 14695 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.934930 11475 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.942169 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.950254 11551 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:30.932829 14909 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:30.958967 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:30.959645 11550 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.929738 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:30.947526 12113 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:16:30.946691 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.947814 11544 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:30.960795 11474 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:30.916952 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 540048us
W20260709 13:16:30.938616 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 239385us
I20260709 13:16:30.964596 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.949383 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1902) took 2864 ms. Trace:
I20260709 13:16:30.929468 14979 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:30.929275 15072 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:16:30.926781 14954 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.969866 15078 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.971045 15037 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:30.971318 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:30.931284 14862 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:30.966073 14857 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:30.992532 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:30.951529 14702 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:30.971640 11474 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:30.988327 11367 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:30.996323 15078 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:30.997946 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:30.999426 15078 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:30.999607 15059 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:16:30.999845 14979 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:31.000267 14979 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:31.000336 14702 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:31.000504 15065 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:16:31.000694 14979 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.006706 11475 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:16:31.008495 11475 pending_rounds.cc:85] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 7
I20260709 13:16:31.008795 11475 pending_rounds.cc:107] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:30.939045 14921 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.970942 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:30.946689 15042 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:30.965319 11545 rpcz_store.cc:276] 0709 13:16:28.084564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.537987 (+453423us) service_pool.cc:224] Handling call
0709 13:16:28.538254 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.538621 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:30.704986 (+2166365us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.946479 (+241493us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.946529 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:30.946569 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.949244 (+  2675us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.949324 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1831561,"prepare.queue_time_us":1859820,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":186,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":3446,"thread_start_us":575431,"threads_started":2}
I20260709 13:16:30.997083 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:30.969174 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:31.010267 11550 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:31.010839 11475 raft_consensus.cc:2889] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:31.007211 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.013566 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:31.000356 15078 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.014318 12116 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:31.020773 15073 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:16:31.024806 14909 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.028030 14891 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.028589 11399 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:31.001700 15033 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:16:31.030539 14695 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:31.031513 15070 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:16:31.035143 14954 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.036645 14978 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.039093 15080 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:31.042507 15085 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:16:31.042901 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 72187.8us
I20260709 13:16:31.043262 14857 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:31.043745 14857 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.044139 14857 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.039862 15080 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:31.049893 14789 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:31.050254 14487 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:31.050458 14789 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:31.060714 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:31.057307 11548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.017s	sys 0.031s
W20260709 13:16:31.062538 11548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.019s	sys 0.031s
I20260709 13:16:31.050748 14487 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:31.012209 14921 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:30.997490 15037 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:16:31.083308 14921 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.083509 15037 consensus_queue.cc:1243] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:31.086741 11476 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:31.087975 11476 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:31.088927 11476 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:31.066967 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 149514us
I20260709 13:16:31.091089 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:31.091357 14893 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:31.092242 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:31.073489 15080 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.074388 14863 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:31.095273 14863 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.092715 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.077311 14862 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:31.052525 14867 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:16:31.082621 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2135) took 1920 ms. Trace:
I20260709 13:16:31.107350 11548 rpcz_store.cc:276] 0709 13:16:29.162624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.219716 (+1057092us) service_pool.cc:224] Handling call
0709 13:16:30.220094 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:30.228578 (+  8484us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:30.439757 (+211179us) log.cc:844] Serialized 14 byte log entry
0709 13:16:31.062980 (+623223us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:31.067532 (+  4552us) log.cc:844] Serialized 52 byte log entry
0709 13:16:31.067748 (+   216us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:31.068015 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 13:16:31.073650 (+  5635us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18003f4540. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:31.081207 (+  7557us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18003f4540. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8da94cfd70 00007f8da94c3293 00007f8da94b8cea 00007f8da94b923a 00007f8da9923a2a 00007f8da99330bf 00007f8da9933038
0709 13:16:31.082222 (+  1015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.082325 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:31.082370 (+    45us) raft_consensus.cc:1685] finished
0709 13:16:31.082381 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:31.082565 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":220836,"prepare.queue_time_us":226417,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":3766,"raft.queue_time_us":9556,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":9907,"spinlock_wait_cycles":28134016,"thread_start_us":578833,"threads_started":1,"wal-append.queue_time_us":531797}
I20260709 13:16:31.070979 14487 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.100220 11551 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:31.038373 14685 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.050933 15027 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.111768 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:31.051039 14789 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:31.121484 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 132168us
W20260709 13:16:31.121064 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 130375us
I20260709 13:16:31.107450 15072 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:31.124468 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.058338 14954 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
W20260709 13:16:31.124255 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 79422.8us
W20260709 13:16:31.119462 15055 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.253s	user 0.003s	sys 0.000s
I20260709 13:16:31.125432 12113 raft_consensus.cc:2368] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
W20260709 13:16:31.126623 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 157515us
I20260709 13:16:31.068051 14851 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:16:31.119860 14863 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.128407 14867 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:31.107663 15005 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:16:31.131431 14685 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.155237 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.131462 14857 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.132469 15076 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:31.124979 15072 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.137162 14954 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:31.137674 15028 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:31.139739 11514 leader_election.cc:336] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.035s (SENT)
I20260709 13:16:31.140923 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:31.174397 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:31.127777 15027 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:31.150897 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.151974 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.152256 11441 leader_election.cc:400] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
I20260709 13:16:31.182307 11441 leader_election.cc:403] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:31.182334 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.152657 11516 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.157464 11402 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:31.184022 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.046150 15042 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.163682 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.184571 12116 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.184945 11393 raft_consensus.cc:2410] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:16:31.184826 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.177435 11475 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:31.181875 11395 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:31.185657 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.186429 11551 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:31.186800 11547 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:31.184429 11516 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:31.185346 11542 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:31.177053 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.188822 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.189536 11549 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:31.178874 11473 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:31.194031 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.262569 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:31.265482 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 105249us
I20260709 13:16:31.275517 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.273088 11400 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:31.165889 15072 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.295580 11471 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:31.296166 11471 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:31.148031 15081 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:16:31.287506 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 103162us
I20260709 13:16:31.289485 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.287541 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:31.287362 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.290962 11396 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:31.292312 15079 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.156129 14777 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 13:16:31.328915 11400 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:16:31.288476 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.188357 14685 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.166213 14954 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:31.173228 14863 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.286691 15027 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:31.309405 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 302723us
I20260709 13:16:31.162276 15091 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.167s
I20260709 13:16:31.344520 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.346472 11367 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.309405 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.348079 11397 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:31.157425 15048 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:16:31.156327 14862 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 13:16:31.332242 15079 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.350122 14862 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:31.350548 14862 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.151741 14257 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.125s
I20260709 13:16:31.351521 15079 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.352730 15098 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:31.192646 14858 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:31.354024 14858 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:31.354242 11476 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.355232 14858 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.192437 15078 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:31.358173 15078 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.336642 14966 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.359998 14966 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.360528 15078 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.362025 11476 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:16:31.362527 11476 pending_rounds.cc:85] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 9
I20260709 13:16:31.293370 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.363267 11469 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:31.327077 12112 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 663) took 1317 ms. Trace:
I20260709 13:16:31.364013 14813 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:31.364051 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.364189 14954 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.364630 14921 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.365139 11469 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:31.365749 14954 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:31.366067 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.366565 14739 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.284997 15089 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:31.370213 15089 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.371968 14857 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.262123 14978 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
I20260709 13:16:31.336072 11398 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:31.371712 15089 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.364229 12112 rpcz_store.cc:276] 0709 13:16:30.009928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.027248 (+ 17320us) service_pool.cc:224] Handling call
0709 13:16:30.051346 (+ 24098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:30.107637 (+ 56291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.107666 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:30.201086 (+ 93420us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b2c000476c8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:30.201482 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 13:16:30.436610 (+235128us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:30.436682 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.436779 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:31.095723 (+658944us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:31.326815 (+231092us) raft_consensus.cc:1685] finished
0709 13:16:31.326837 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:31.327012 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1313,"raft.queue_time_us":1410,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":9223808,"thread_start_us":234702,"threads_started":1,"wal-append.queue_time_us":574172}
I20260709 13:16:31.311352 11471 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.381927 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.382655 11469 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:31.287537 15061 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:31.309201 12118 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:31.303606 14971 raft_consensus.cc:2764] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:16:31.362349 14966 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:31.383558 11476 pending_rounds.cc:107] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:16:31.384115 11473 raft_consensus.cc:2393] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:31.384968 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.385236 11469 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:31.339484 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.262045 14969 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.298411 15016 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:16:31.390332 11476 raft_consensus.cc:2889] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:31.438879 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92503.9us
I20260709 13:16:31.440439 14874 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:31.426784 15061 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:31.442306 15082 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:16:31.444514 14969 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.343420 14685 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.448012 14994 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:31.449258 14685 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.451385 15099 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:31.433298 15079 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:31.442164 15061 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.454614 15027 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.457134 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 503) took 1084 ms. Trace:
I20260709 13:16:31.460003 11367 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.384725 15083 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:31.366911 14921 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:31.465750 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 176396us
I20260709 13:16:31.464051 12118 rpcz_store.cc:276] 0709 13:16:30.372833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.372945 (+   112us) service_pool.cc:224] Handling call
0709 13:16:30.373204 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:31.389646 (+1016442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.456914 (+ 67268us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54003778d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:31.457078 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148842752}
W20260709 13:16:31.469901 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 168314us
I20260709 13:16:31.465216 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2172) took 1092 ms. Trace:
I20260709 13:16:31.476779 11474 rpcz_store.cc:276] 0709 13:16:30.372801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.395579 (+ 22778us) service_pool.cc:224] Handling call
0709 13:16:31.465032 (+1069453us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540044b1d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:31.465157 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287451008}
I20260709 13:16:31.462726 12116 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:31.480696 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:31.386657 14437 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
W20260709 13:16:31.472294 14648 consensus_meta.cc:287] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.521s	user 0.004s	sys 0.005s
I20260709 13:16:31.370716 14789 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.498253 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.501595 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.495981 14685 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:31.511736 14685 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.512168 14685 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.351548 14862 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.521656 14789 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.483367 15048 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:16:31.523634 15095 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:16:31.472119 14702 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:16:31.434322 14973 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:16:31.536689 14973 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:31.537231 14973 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.544675 15083 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:31.545230 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62237.5us
I20260709 13:16:31.545725 15083 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.545377 15028 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.548722 14487 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.503564 14648 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.604410 14648 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.555708 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.605876 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.606767 11397 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:31.607699 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.608362 11397 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:31.504290 15097 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:31.593132 15029 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:16:31.610669 11297 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 142) took 16636 ms (client timeout 15000 ms). Trace:
I20260709 13:16:31.593946 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.613008 14487 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.595391 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.614933 11399 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:31.616314 11399 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:16:31.616675 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 291685us
I20260709 13:16:31.484383 14977 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:16:31.618242 11365 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:31.596591 14979 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.005s
I20260709 13:16:31.618564 15028 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:31.619236 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.596978 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.618508 11475 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:31.599022 14959 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:16:31.623867 15093 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:16:31.601898 11394 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:31.610960 15029 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:31.626866 12116 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:31.627676 15029 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:31.629004 15100 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.343s	user 0.003s	sys 0.000s
I20260709 13:16:31.628841 15048 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:31.628741 15029 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:31.620280 11297 rpcz_store.cc:269] 0709 13:16:14.974692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.975010 (+   318us) service_pool.cc:224] Handling call
0709 13:16:15.728330 (+753320us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b5400002490. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f50b2 00007f8daf76ea89 00007f8daf706da6 00007f8daf6d8088 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc
0709 13:16:31.610553 (+15882223us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:26.288395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:26.289144 (+   749us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:26.289205 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:16:26.289228 (+    23us) tablet.cc:662] Decoding operations
0709 13:16:26.290576 (+  1348us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:26.290661 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:26.290693 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:16:26.506199 (+215506us) tablet.cc:701] Row locks acquired
0709 13:16:26.506253 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:16:26.506474 (+   221us) write_op.cc:270] Start()
0709 13:16:26.506658 (+   184us) write_op.cc:276] Timestamp: P: 1783602986506402 usec, L: 0
0709 13:16:26.506684 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:16:26.532641 (+ 25957us) log.cc:844] Serialized 5076 byte log entry
0709 13:16:26.823175 (+290534us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:27.275563 (+452388us) op_driver.cc:464] REPLICATION: finished
0709 13:16:27.455623 (+180060us) write_op.cc:301] APPLY: starting
0709 13:16:27.723658 (+268035us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:27.807762 (+ 84104us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:27.817448 (+  9686us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:28.046413 (+228965us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:28.046821 (+   408us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:16:28.046886 (+    65us) write_op.cc:312] APPLY: finished
0709 13:16:28.047375 (+   489us) log.cc:844] Serialized 95 byte log entry
0709 13:16:29.953134 (+1905759us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da680f0ac 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b
0709 13:16:30.148963 (+195829us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:30.149104 (+   141us) write_op.cc:454] Released schema lock
0709 13:16:30.149172 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:30.230799 (+ 81627us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:30.234375 (+  3576us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":290339,"reported_tablets":24,"spinlock_wait_cycles":4311296,"child_traces":[["op",{"apply.queue_time_us":156626,"apply.run_cpu_time_us":42791,"apply.run_wall_time_us":2765569,"num_ops":13,"prepare.queue_time_us":239012,"prepare.run_cpu_time_us":32122,"prepare.run_wall_time_us":483427,"raft.queue_time_us":363690,"raft.run_cpu_time_us":31895,"raft.run_wall_time_us":185024,"replication_time_us":723146,"spinlock_wait_cycles":40573568,"thread_start_us":2715168,"threads_started":5,"wal-append.queue_time_us":2389095,"wal-append.run_cpu_time_us":10915,"wal-append.run_wall_time_us":2399884}]]}
I20260709 13:16:31.633613 15072 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.628842 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.538362 14973 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.637297 14973 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.636379 15072 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.619258 14979 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.639603 15027 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.640345 14979 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.640260 15027 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.546198 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.644011 11543 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:31.647715 14921 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:31.649121 14909 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:31.649600 14909 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:31.650658 14909 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.651762 11471 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:31.652093 14909 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.663427 14695 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:31.668310 14695 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:31.668877 14695 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.669734 14695 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.670966 14695 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.672375 14773 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:31.672636 14862 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.673090 15092 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.673453 14979 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:31.673853 14979 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.673629 15092 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.667557 15042 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:31.675034 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:31.666558 14977 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:31.664902 15082 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:31.676606 14977 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:31.676815 14959 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:31.677351 14959 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.677294 14921 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:31.677893 14959 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.677613 14977 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.678193 15092 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:31.674718 14979 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.678965 14977 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.676703 15082 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:31.681208 15082 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.682380 15082 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.676569 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.683867 14773 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.684355 14979 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.687443 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.687734 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.695958 12118 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:31.696610 15092 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.697041 15080 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:31.697566 15080 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:31.697307 15092 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:31.698382 15080 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:31.701712 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 155631us
I20260709 13:16:31.702656 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:31.704038 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:31.704389 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86235us
I20260709 13:16:31.685717 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.707609 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:31.484603 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:31.708748 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 224227us
I20260709 13:16:31.710928 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.725924 11543 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:31.745739 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:31.751088 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.744442 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:31.756373 11401 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:31.757335 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 280434us
I20260709 13:16:31.714088 14773 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:31.752877 14937 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:16:31.714263 15097 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.713757 14867 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:31.738327 14437 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:31.784874 11393 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:31.786228 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.752487 14971 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:31.725284 15042 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:16:31.773418 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.800873 11542 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.801759 11551 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:31.774127 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.779582 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.780174 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.803303 11548 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:31.752390 14799 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:31.796489 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.781055 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.808812 11547 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:31.809223 11544 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.788090 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.766139 14813 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:31.760673 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46136.9us
I20260709 13:16:31.789716 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:31.806826 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:31.808126 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 49341.6us
I20260709 13:16:31.810088 11549 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.725924 14921 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.811650 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.755143 15016 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:16:31.820789 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:31.832907 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:31.790925 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.780395 14701 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:31.821250 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.821892 11473 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:16:31.822198 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.822458 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.822520 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.831372 12116 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:16:31.831825 11401 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:31.785286 15027 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:31.843555 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.869761 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:31.821197 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.871539 11542 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:31.864250 11473 pending_rounds.cc:85] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 10
I20260709 13:16:31.873040 11473 pending_rounds.cc:107] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:16:31.873318 11473 raft_consensus.cc:2889] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:31.752406 15103 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:16:31.893625 11402 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:31.835569 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:31.896101 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:31.784996 15092 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:31.791356 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:31.852844 15113 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:16:31.820163 11475 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:31.845314 11395 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:31.836445 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.884326 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.873591 14701 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.886905 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.887156 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:31.943713 11397 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.888060 11545 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:31.944669 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.944990 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.890812 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:31.898440 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.898429 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:31.946444 11544 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:31.947834 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.950165 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:31.957770 11476 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:31.959007 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:31.959794 11472 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.909547 11549 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:31.783905 15029 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:16:31.816550 14813 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:31.864358 15108 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:31.824312 11551 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:31.983618 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:31.852298 15125 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:16:31.914229 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.919045 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:31.820024 14437 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:31.993818 11399 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:31.822755 14867 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:31.966238 11401 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:31.967881 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:31.851114 14799 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:31.996018 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.980162 11472 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:31.874508 14239 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:31.826462 14773 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:16:31.995407 11549 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:31.922178 14966 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.981940 14813 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:32.001865 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 56106.8us
I20260709 13:16:32.003400 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:31.816711 14971 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:32.004822 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:16:32.054945 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50169.6us
I20260709 13:16:31.911857 14954 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:32.013424 15129 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.013623 14648 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:32.013779 14799 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.008772 14789 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:16:32.014531 11298 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.015443 14976 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.015722 15089 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.016094 15114 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.016217 14994 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:32.024739 14701 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.032874 14437 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:31.969929 14858 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:16:32.039669 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.040714 14239 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.049549 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:32.104269 11544 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
W20260709 13:16:32.063534 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.064854 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 253226us
I20260709 13:16:32.069031 11367 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:32.056383 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 53081.5us
I20260709 13:16:32.004467 14257 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.005352 15086 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:32.099068 12113 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:32.104429 11397 raft_consensus.cc:606] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.111622 11397 raft_consensus.cc:614] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:32.103717 11298 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:32.071444 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.107664 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:32.113307 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44371.3us
I20260709 13:16:32.114369 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.114814 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:32.108593 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.118741 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:32.101703 14685 consensus_meta.cc:287] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.001s
I20260709 13:16:32.092474 14893 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:32.126164 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 143319us
I20260709 13:16:32.138018 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.004458 14867 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.094154 15029 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:32.115942 14239 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.103426 15114 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.105212 15089 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.095652 14954 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.151189 14685 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.055699 14966 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.099217 14976 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.116145 14257 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.093863 15126 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:32.136619 14789 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:32.161540 14893 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.117460 15080 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:32.173807 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65460us
W20260709 13:16:32.215543 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77656us
I20260709 13:16:32.337328 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:32.267324 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 137332us
I20260709 13:16:32.268493 15079 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.136832 15106 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:16:32.124332 15078 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.130004 14874 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:32.132463 14977 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:32.159658 15099 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:32.285948 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.287029 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 336892us
I20260709 13:16:32.267438 11298 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.300554 11549 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:16:32.398104 11470 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:16:32.287635 15055 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.168s	user 0.004s	sys 0.004s
W20260709 13:16:32.294158 14973 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.656s	user 0.019s	sys 0.000s
I20260709 13:16:32.289758 15086 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:32.290818 15029 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:32.312806 14959 consensus_meta.cc:287] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.003s
I20260709 13:16:32.285918 15133 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:32.337570 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:32.454978 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 117478us
I20260709 13:16:32.455343 11298 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.456501 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:32.457918 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.387307 12123 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:16:32.399221 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61972.8us
I20260709 13:16:32.459491 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:32.311196 15131 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.316331 15096 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.421854 12118 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:32.344566 15080 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:32.313709 14876 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:16:32.290132 15139 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:16:32.404875 15099 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.474143 11298 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.492022 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:32.495961 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.460888 15029 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:32.506587 12118 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:32.464607 15133 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.472476 14977 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.477757 14487 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:32.480820 14777 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:32.508709 11298 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.490507 15137 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:32.461419 15086 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.492623 15099 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:32.472615 14959 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.476811 15096 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.519761 15129 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:16:32.519611 15137 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:32.533296 15080 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.539219 15137 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:32.539649 14867 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:32.539928 15080 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.514724 15146 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:16:32.476948 15131 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.543959 12119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:56142
I20260709 13:16:32.544625 12119 raft_consensus.cc:606] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.545315 15131 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.477924 15140 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:16:32.547984 15089 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.550340 14685 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:32.551123 15128 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:16:32.551842 14966 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.552368 14257 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.532246 15092 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:16:32.553097 15092 consensus_queue.cc:1243] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:32.553339 15114 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.554699 15096 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.556012 14954 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.556145 14979 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:32.556529 14979 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:32.556954 14976 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.557360 14979 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.557857 15146 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:32.558368 15146 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:32.559193 15146 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.560380 15146 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:32.564260 14959 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:32.565075 14862 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.892s	user 0.019s	sys 0.005s
I20260709 13:16:32.564946 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.565933 11542 raft_consensus.cc:2410] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:32.565958 12117 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:32.566704 15082 consensus_meta.cc:287] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.884s	user 0.007s	sys 0.010s
W20260709 13:16:32.567672 14813 consensus_meta.cc:287] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
I20260709 13:16:32.568084 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:32.568907 14695 consensus_meta.cc:287] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.897s	user 0.011s	sys 0.000s
I20260709 13:16:32.569780 11551 raft_consensus.cc:2410] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:16:32.571707 11470 raft_consensus.cc:2410] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:32.512568 15133 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:32.520331 15079 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:32.596477 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.558245 14239 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.600634 11401 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:32.600770 15139 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.518694 14977 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.603999 15086 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.618767 11474 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:32.623812 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:32.627184 11298 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:32.631183 11473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.014s	sys 0.017s
W20260709 13:16:32.647843 11473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.014s	sys 0.018s
W20260709 13:16:32.631623 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.013s	sys 0.022s
W20260709 13:16:32.649159 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.015s	sys 0.022s
I20260709 13:16:32.636132 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.642858 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:32.649530 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 530653us
I20260709 13:16:32.654569 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:32.658314 11298 catalog_manager.cc:5697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.528278 14789 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 13:16:32.657223 15076 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:16:32.683314 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.693851 15114 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:32.694082 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.694818 11401 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:32.702042 15099 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.568142 14813 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.747437 11547 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:32.725019 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:32.740037 14893 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.648825 15139 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.752525 15139 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:32.743378 15137 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.755290 15137 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:32.756421 15082 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.758240 15082 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:32.760994 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.761219 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:32.761937 11471 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:32.762284 11473 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:32.763947 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:32.765625 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 299178us
I20260709 13:16:32.766137 11298 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 10 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:32.768138 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.771778 12115 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:32.771207 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:32.740128 14977 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.718634 15029 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.783516 11298 catalog_manager.cc:5697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:32.785971 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 328099us
W20260709 13:16:32.786684 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.790661 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.790452 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:32.793248 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.793524 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:32.672214 15086 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:16:32.795275 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.796418 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.775521 15082 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.799321 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:32.797341 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.802772 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.803145 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:32.808310 11394 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:32.806475 11401 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:32.774765 15153 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.777921 15126 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:32.817303 11298 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.820370 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:32.828513 11395 raft_consensus.cc:606] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.799237 15140 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:32.833055 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 355602us
I20260709 13:16:32.833958 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:32.821014 14937 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.838757 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.820825 14977 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.824765 15114 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.839658 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.861235 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.821240 15154 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.829725 15153 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.837271 15105 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:16:32.839183 14867 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:32.842721 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:32.843281 15042 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:32.830152 15092 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.822198 15082 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.848749 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:32.855472 14893 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.863997 11298 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.862736 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.860069 14973 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.831166 14789 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:16:32.858018 14813 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.879998 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.891955 15099 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:32.892158 11551 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:32.893196 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.893757 11551 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:32.894061 15086 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:32.839332 15140 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.894589 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.883805 11298 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.895192 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1930) took 1108 ms. Trace:
I20260709 13:16:32.869244 15154 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.879624 15076 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.872058 15114 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.881781 14893 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.886862 15092 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.894618 15086 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:32.884086 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.867460 14648 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.870940 15078 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:32.908291 11298 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.859611 14937 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.900025 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.873133 15042 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:32.903463 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:32.864531 14977 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:32.897943 11550 rpcz_store.cc:276] 0709 13:16:31.786853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.787180 (+   327us) service_pool.cc:224] Handling call
0709 13:16:31.816082 (+ 28902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:32.145518 (+329436us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b18002c7280. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8da9911060 00007f8da9911bfb 00007f8da991c64b
0709 13:16:32.268307 (+122789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.268367 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:32.284041 (+ 15674us) log.cc:844] Serialized 26 byte log entry
0709 13:16:32.776314 (+492273us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:32.776349 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:32.776473 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:32.894882 (+118409us) raft_consensus.cc:1685] finished
0709 13:16:32.894902 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:32.895137 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34824,"raft.queue_time_us":127263,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":16510,"spinlock_wait_cycles":63394176,"thread_start_us":562407,"threads_started":2,"wal-append.queue_time_us":580451}
I20260709 13:16:32.928301 15126 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.902643 14648 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:32.904086 14893 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.898728 14973 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.893977 15153 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.935519 11298 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.902437 15137 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.869087 15166 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:16:32.901022 15072 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.942328 11551 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:32.957902 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54542.8us
I20260709 13:16:32.968896 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:32.957477 15078 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:32.984005 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:32.966096 15158 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.410s	user 0.000s	sys 0.009s
I20260709 13:16:32.991468 15140 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:32.980363 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.911675 14867 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:32.966441 14648 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.982960 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:32.994653 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 167597us
I20260709 13:16:32.997521 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.984264 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.989399 11469 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:32.998769 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:32.998967 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.992914 11473 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:32.966185 15106 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:32.992815 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.003662 11550 raft_consensus.cc:2410] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:32.972131 11298 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 8, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.000519 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:32.995170 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:33.002940 11395 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:16:33.007015 11471 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:33.009001 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.009696 11469 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:33.010022 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2163) took 1121 ms. Trace:
I20260709 13:16:33.011581 11544 rpcz_store.cc:276] 0709 13:16:31.888874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.105205 (+216331us) service_pool.cc:224] Handling call
0709 13:16:32.135392 (+ 30187us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:32.278418 (+143026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.278456 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:32.301053 (+ 22597us) log.cc:844] Serialized 52 byte log entry
0709 13:16:32.395224 (+ 94171us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:32.408396 (+ 13172us) log.cc:844] Serialized 14 byte log entry
0709 13:16:32.453106 (+ 44710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:32.453209 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:32.896790 (+443581us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:33.003666 (+106876us) raft_consensus.cc:1685] finished
0709 13:16:33.009811 (+  6145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.009954 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199576,"prepare.queue_time_us":117775,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2824,"raft.queue_time_us":201874,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":21480,"spinlock_wait_cycles":4415232,"thread_start_us":51423,"threads_started":1,"wal-append.queue_time_us":170283}
I20260709 13:16:32.987771 15029 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.014619 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.014556 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.019270 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 9, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.020154 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.022804 11470 raft_consensus.cc:2393] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:33.024982 11298 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:32.975232 15137 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:32.967109 15133 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.003197 15106 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.031941 11298 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.009977 14954 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.037672 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.038390 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.038605 11470 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:33.034271 14973 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.008157 14867 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:32.989851 15072 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.016893 15146 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.016392 14685 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:33.059477 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64387us
I20260709 13:16:33.060446 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:33.056746 11298 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:33.028544 15029 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.062347 11550 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:33.063597 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.064976 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:33.004055 15086 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.037665 15106 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:33.076424 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2168) took 1186 ms. Trace:
I20260709 13:16:33.093402 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1516) took 1143 ms. Trace:
I20260709 13:16:33.129319 11393 rpcz_store.cc:276] 0709 13:16:31.949865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.950025 (+   160us) service_pool.cc:224] Handling call
0709 13:16:31.950345 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.333495 (+383150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.333530 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:32.333574 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:32.333605 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.093148 (+759543us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b540014b4d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.093198 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.093330 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880709376}
W20260709 13:16:33.129976 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 65059.3us
I20260709 13:16:33.119372 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.121553 15072 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.121722 15042 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.062860 15146 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.137241 15042 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.133924 14937 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.138559 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 679) took 1089 ms. Trace:
I20260709 13:16:33.105393 15056 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:33.139066 14937 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:33.139639 15056 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:33.140718 15184 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:33.131104 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 415665us
I20260709 13:16:33.130887 11543 rpcz_store.cc:276] 0709 13:16:31.890261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.603482 (+713221us) service_pool.cc:224] Handling call
0709 13:16:32.603868 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:32.604328 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:32.605566 (+  1238us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.012224 (+406658us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.012659 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.012793 (+   134us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:33.022403 (+  9610us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.034902 (+ 12499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.035018 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.076223 (+ 41205us) raft_consensus.cc:1685] finished
0709 13:16:33.076247 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.076369 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51578,"prepare.queue_time_us":2005,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4295,"raft.queue_time_us":141341,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":35062,"thread_start_us":308365,"threads_started":1,"wal-append.queue_time_us":411314}
W20260709 13:16:33.098958 15164 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:33.051178 15114 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.058615 15139 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.139024 12123 rpcz_store.cc:276] 0709 13:16:32.049213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.049379 (+   166us) service_pool.cc:224] Handling call
0709 13:16:32.049756 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.458853 (+409097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.138309 (+679456us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400720454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.138498 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674138496}
I20260709 13:16:33.140964 15056 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.062788 15082 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.130505 14977 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.075330 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2166) took 1185 ms. Trace:
I20260709 13:16:33.179322 15139 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.180282 14648 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.153458 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.174000 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.073375 15105 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.142900 15146 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.157505 15154 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.145350 14799 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:33.164238 15171 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
W20260709 13:16:33.164987 15174 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:16:33.133355 15167 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:16:33.147576 15184 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.167297 15133 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.102758 14867 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.167986 11298 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.083639 15029 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.171195 11471 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:33.087801 15078 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:33.131791 15081 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.195s	user 0.018s	sys 0.013s
I20260709 13:16:33.174898 11395 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:33.180127 11549 rpcz_store.cc:276] 0709 13:16:31.889635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.397979 (+508344us) service_pool.cc:224] Handling call
0709 13:16:32.398239 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.398425 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:32.832660 (+434235us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.075017 (+242357us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:33.075065 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:33.075110 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.075170 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.075276 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77757,"prepare.queue_time_us":129559,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1989,"raft.queue_time_us":10790,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":1533,"thread_start_us":656048,"threads_started":2}
I20260709 13:16:33.218228 11298 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.213219 11394 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:16:33.078325 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 258961us
I20260709 13:16:33.177973 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2157) took 1295 ms. Trace:
I20260709 13:16:33.222193 11546 rpcz_store.cc:276] 0709 13:16:31.882097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.883694 (+  1597us) service_pool.cc:224] Handling call
0709 13:16:31.883932 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.112283 (+228351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.112310 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:32.112347 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:32.112375 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.177753 (+1065378us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400404f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.177794 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.177905 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524713728}
I20260709 13:16:33.223423 11298 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.192550 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:33.175902 15136 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.059s	user 0.006s	sys 0.005s
I20260709 13:16:33.166198 15137 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.225989 11298 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.192039 14799 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.227540 14799 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.227977 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 508) took 1446 ms. Trace:
I20260709 13:16:33.156494 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.229066 11397 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:33.228385 11398 rpcz_store.cc:276] 0709 13:16:31.782014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.782393 (+   379us) service_pool.cc:224] Handling call
0709 13:16:31.813856 (+ 31463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:31.814164 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:31.814190 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:31.824753 (+ 10563us) log.cc:844] Serialized 26 byte log entry
0709 13:16:31.952306 (+127553us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:31.952363 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.952466 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.174902 (+1222436us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:33.227673 (+ 52771us) raft_consensus.cc:1685] finished
0709 13:16:33.227696 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.227905 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13007,"spinlock_wait_cycles":51819136,"thread_start_us":29332,"threads_started":1,"wal-append.queue_time_us":131192}
I20260709 13:16:33.231940 11367 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:33.232800 14874 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:33.234087 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.234902 14874 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:33.235348 14874 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:33.228641 11298 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 8, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.235743 14874 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.235503 14799 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.236769 11298 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.238668 14799 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.238619 11298 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.240890 11298 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 7, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.242929 11298 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 7, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.244679 11298 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.245913 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.247486 15177 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:33.247995 15177 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.248394 15177 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.253276 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:33.255240 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:33.245968 11298 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 6, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.258507 11400 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:33.263585 11470 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:16:33.264060 11470 pending_rounds.cc:85] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 7
I20260709 13:16:33.263945 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.264484 11470 pending_rounds.cc:107] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:16:33.264811 11470 raft_consensus.cc:2889] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:33.221271 15081 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.019s	sys 0.013s
I20260709 13:16:33.208693 15184 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:33.277132 11298 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 2 to 6, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.276244 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:33.283715 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 152471us
I20260709 13:16:33.279333 11298 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 7, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 7 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:16:33.292042 11298 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.302353 11549 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:33.309340 11546 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:33.318691 11546 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:33.324734 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.286065 14874 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.330240 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.334829 15106 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.336030 11476 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:33.337222 15106 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.296869 15105 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:16:33.339223 15029 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.339555 15105 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:33.340008 15105 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:33.328227 15072 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.341058 15105 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.342509 15105 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:33.340037 14648 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.295779 14893 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.345786 14893 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.338903 14874 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.348486 11544 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:33.349575 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:33.353152 15072 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.337260 15177 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.357663 15177 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.314821 15139 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.358235 15072 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:33.382015 15072 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.382772 15072 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:33.389078 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.391400 15177 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.398008 15177 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.400132 15177 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.400727 14979 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.401850 11472 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:33.402210 11472 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:33.380823 15081 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:33.392889 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:33.391587 15188 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.402976 11472 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.412261 15200 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:33.403200 15072 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.408792 15078 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:33.420163 15200 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:33.411475 15081 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.445366 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.446053 11468 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:33.446400 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2189) took 1501 ms. Trace:
I20260709 13:16:33.446782 11475 rpcz_store.cc:276] 0709 13:16:31.945501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.956284 (+ 10783us) service_pool.cc:224] Handling call
0709 13:16:31.956639 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:32.338467 (+381828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.338511 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:32.404769 (+ 66258us) log.cc:844] Serialized 52 byte log entry
0709 13:16:32.785273 (+380504us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:32.785332 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.446000 (+660668us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400429ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.446048 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.446095 (+    47us) raft_consensus.cc:1685] finished
0709 13:16:33.446108 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.446341 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6945,"raft.queue_time_us":74630,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":38701,"spinlock_wait_cycles":857055616,"thread_start_us":312905,"threads_started":1,"wal-append.queue_time_us":425611}
I20260709 13:16:33.449725 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.450466 11475 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:33.451308 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.412096 15188 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.452521 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.453150 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2153) took 1580 ms. Trace:
I20260709 13:16:33.453569 11547 rpcz_store.cc:276] 0709 13:16:31.872347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.872489 (+   142us) service_pool.cc:224] Handling call
0709 13:16:31.872764 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.749692 (+876928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.452945 (+703253us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540033d6d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.453096 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010989440}
I20260709 13:16:33.454531 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:33.455353 15129 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:33.455760 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.455739 15129 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.456593 15139 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.455368 15081 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.456617 15129 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:33.461810 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.462858 11469 raft_consensus.cc:2410] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:16:33.439427 15200 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.456068 15188 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.435338 15078 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.453125 11475 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:33.467628 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.466145 11468 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:33.497418 11471 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:33.497815 14954 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:33.499816 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.458830 14701 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:16:33.463529 15139 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:16:33.488530 15200 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.490484 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2192) took 1532 ms. Trace:
I20260709 13:16:33.495721 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2171) took 1197 ms. Trace:
I20260709 13:16:33.506050 11544 rpcz_store.cc:276] 0709 13:16:32.298742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.029319 (+730577us) service_pool.cc:224] Handling call
0709 13:16:33.055873 (+ 26554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.348895 (+293022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.495504 (+146609us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54003e9ed4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.495665 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672525184}
I20260709 13:16:33.464846 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.504791 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.505331 11545 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:16:33.505992 11477 rpcz_store.cc:276] 0709 13:16:31.957748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.963111 (+  5363us) service_pool.cc:224] Handling call
0709 13:16:31.963472 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:31.963731 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:31.964097 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.489973 (+1525876us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b18002f7fa0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a
0709 13:16:33.490156 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:33.490203 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:33.490245 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.490308 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.490416 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17883,"prepare.queue_time_us":31043,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":773,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":48126336,"thread_start_us":109302,"threads_started":1,"wal-append.queue_time_us":311792,"wal-append.run_cpu_time_us":1122,"wal-append.run_wall_time_us":1001148}
I20260709 13:16:33.501541 15126 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.512761 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.505923 15020 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.507131 15042 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:33.512526 14813 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:33.540042 15139 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:33.544744 15139 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:33.545483 15200 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.541913 14773 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:33.542689 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.536725 15072 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.545603 15139 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.548198 15078 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.547158 11469 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:33.535578 14867 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:33.549593 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:33.551043 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.517292 14921 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:16:33.543830 15020 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.550048 15042 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.551739 15188 raft_consensus.cc:993] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:16:33.589793 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.590507 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.591363 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.592662 11477 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:33.586239 14685 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.596763 14685 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.600575 15042 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.605724 15020 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.611909 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:33.613055 15042 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:33.613636 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:33.583707 15208 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:16:33.615367 14648 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.613749 15042 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.558064 15078 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:33.623266 11515 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:33.612939 15129 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.614825 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:33.572366 11469 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:33.624430 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:33.545665 15184 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.630966 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.650796 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.563803 14867 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.550179 14813 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.594061 15188 raft_consensus.cc:1081] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:16:33.584275 14909 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:16:33.650065 14979 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.647568 15106 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.626430 14685 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.622246 15072 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.627370 14701 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.654448 15126 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.666158 15137 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:33.655783 15091 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.654884 14648 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:33.647568 15029 raft_consensus.cc:2764] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:16:33.657689 14799 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:33.677579 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2170) took 1559 ms. Trace:
I20260709 13:16:33.711783 11550 rpcz_store.cc:276] 0709 13:16:32.117793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.006151 (+888358us) service_pool.cc:224] Handling call
0709 13:16:33.006426 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.074242 (+ 67816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.677342 (+603100us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b54002c65d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.677522 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128175488}
I20260709 13:16:33.666066 14813 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.673465 14979 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.671730 15133 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:33.627218 15078 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.719506 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.700121 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.713529 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.718186 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.728334 11516 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.729653 11402 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:33.728647 15078 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.734937 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.735545 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.735771 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.736507 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.736104 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.736878 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.737097 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.741423 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.743698 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.691624 15072 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.744333 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:33.691824 14701 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.750551 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.694844 15126 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.754387 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:33.758473 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 99519.6us
I20260709 13:16:33.762641 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:33.765275 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.765585 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.766765 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.770004 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:33.770550 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:33.771133 11545 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:33.744289 14858 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:33.774283 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:33.708721 15091 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.775295 12118 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:33.777299 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.715255 14648 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.779426 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:33.778244 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.744818 15097 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:33.785034 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.781399 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.781670 11402 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:33.786545 11544 raft_consensus.cc:2410] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:16:33.783820 11547 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:33.758195 15208 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:33.789865 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61628us
I20260709 13:16:33.769574 15139 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:33.797622 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92409.1us
I20260709 13:16:33.793606 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:33.745692 14773 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:16:33.774887 14858 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:33.796069 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.807531 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.801267 11399 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:33.804100 11545 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:33.812628 15184 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.822649 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.823699 11393 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:33.830454 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.831146 11544 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
W20260709 13:16:33.830823 11542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2154) took 1949 ms (client timeout 1832 ms). Trace:
W20260709 13:16:33.831928 11542 rpcz_store.cc:269] 0709 13:16:31.881181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.890422 (+  9241us) service_pool.cc:224] Handling call
0709 13:16:33.830571 (+1940149us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001a6dd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:33.830741 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425679104}
I20260709 13:16:33.833209 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.833776 11542 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:33.834347 15092 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.835265 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.834653 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.836939 11542 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:33.837272 11475 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:33.836546 11546 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:33.838375 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:33.839892 11551 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:16:33.840271 11551 pending_rounds.cc:85] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 6
I20260709 13:16:33.840603 11551 pending_rounds.cc:107] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:16:33.840883 11551 raft_consensus.cc:2889] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:33.842519 15214 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:33.843806 11471 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:33.834893 15092 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.845350 15092 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:33.847451 15092 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.836521 15042 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.849452 14863 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:33.834803 15129 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.791491 15200 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:33.855218 11469 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:33.859534 14858 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.860822 14858 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:33.857571 14971 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.864703 14648 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.866307 15072 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.867270 14979 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.867362 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:33.849964 14863 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.852897 15139 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:33.869779 15126 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.870096 15139 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.870513 15139 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:33.872756 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2188) took 2000 ms. Trace:
I20260709 13:16:33.873133 11474 rpcz_store.cc:276] 0709 13:16:31.872428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.872945 (+   517us) service_pool.cc:224] Handling call
0709 13:16:31.873334 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:32.647424 (+774090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.872509 (+1225085us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54002a4754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:33.872693 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084312320}
I20260709 13:16:33.856053 14867 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.799427 15208 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.868321 11542 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:33.877044 15208 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.877329 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:33.877724 12123 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:33.878134 11542 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:33.793166 15097 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.880156 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:33.869171 14701 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:33.881894 15097 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.868722 15091 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.893797 11474 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:33.894665 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.899191 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.904018 11396 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:16:33.910548 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:33.908250 11471 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:33.906492 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.914325 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.920543 12085 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.921463 15188 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:33.922786 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:33.930878 14921 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.931447 14921 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:33.933501 14921 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:33.936627 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:33.937505 11470 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:33.960893 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:33.971521 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.953704 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:33.975672 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:33.903519 14976 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:34.036628 11474 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:16:34.041658 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:34.099675 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.914094 15126 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:34.125459 11470 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:34.126252 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.126969 12112 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:33.927953 14701 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:33.913446 14863 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:33.926412 15016 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:16:34.144506 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 183691us
I20260709 13:16:34.158339 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:34.163247 11474 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:34.174943 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:34.197757 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.212646 15133 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:34.128991 15097 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.150823 15215 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:34.226117 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:34.268468 11470 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:34.150441 14773 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:34.274735 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 480668us
I20260709 13:16:34.152233 14921 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:34.224617 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66403.3us
I20260709 13:16:34.163276 15188 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:34.154225 15079 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.199263 15219 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.311861 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.337303 11542 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:34.314669 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:34.220386 15187 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.352698 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:34.245565 14857 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:34.249115 14863 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.154603 15126 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.156543 14976 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.359930 11472 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:34.364506 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:34.369630 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.383919 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:34.387142 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 575) took 2328 ms. Trace:
I20260709 13:16:34.387792 12117 rpcz_store.cc:276] 0709 13:16:32.058694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.058846 (+   152us) service_pool.cc:224] Handling call
0709 13:16:32.059349 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.566750 (+507401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.386804 (+1820054us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540021dc54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:34.387056 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164719872}
I20260709 13:16:34.307622 15218 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:16:34.360050 15199 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:16:34.420887 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.358319 14851 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:34.320622 14773 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.213835 15133 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.448906 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.451865 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.452603 15126 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:34.453333 14858 consensus_meta.cc:287] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.000s
I20260709 13:16:34.329710 14921 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.358265 14857 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:34.455478 14857 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:34.456037 14921 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:34.456679 12119 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
W20260709 13:16:34.458534 15139 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.588s	user 0.006s	sys 0.002s
I20260709 13:16:34.458971 15139 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.359025 15219 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.463203 15219 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.375535 15187 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.466259 14857 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.466969 14976 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.467238 15129 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:34.468246 15221 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:16:34.355291 15079 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.374467 14863 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.332415 15016 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:16:34.361022 15220 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:34.469910 15129 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.267089 15097 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.478675 12113 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:34.477035 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 470) took 1461 ms. Trace:
I20260709 13:16:34.479908 11543 rpcz_store.cc:276] 0709 13:16:33.015313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.307211 (+291898us) service_pool.cc:224] Handling call
0709 13:16:33.307567 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.307942 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.307981 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.315424 (+  7443us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.814314 (+498890us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18003cb020. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:33.980547 (+166233us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:33.980600 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.980673 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.473666 (+492993us) raft_consensus.cc:1685] finished
0709 13:16:34.473697 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.476958 (+  3261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5580,"raft.queue_time_us":3752,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":16063,"spinlock_wait_cycles":4983680,"thread_start_us":165221,"threads_started":1,"wal-append.queue_time_us":908869}
I20260709 13:16:34.481128 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.482483 11543 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:34.483829 11542 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:34.450856 14773 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:34.458444 14813 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.729s	user 0.003s	sys 0.005s
I20260709 13:16:34.466800 15187 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.514810 15139 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.528069 15129 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:34.374226 11441 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:34.568370 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 194084us
W20260709 13:16:34.583765 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.584731 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:34.593220 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:34.596081 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2194) took 1593 ms. Trace:
I20260709 13:16:34.596750 11476 rpcz_store.cc:276] 0709 13:16:33.002513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.015344 (+ 12831us) service_pool.cc:224] Handling call
0709 13:16:33.015597 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.336462 (+320865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.595495 (+1259033us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400213c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:34.595852 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736497536}
I20260709 13:16:34.603219 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:34.604467 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 324644us
I20260709 13:16:34.606330 11542 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:34.608636 11542 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:34.610311 11542 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:34.680970 15107 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:16:34.683822 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.693487 11544 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:16:34.694775 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.698180 14857 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.699584 14257 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:34.700663 14257 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.704624 12116 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:34.705799 14739 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.708900 14863 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.709425 14487 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.710532 14487 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.712993 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.713636 14487 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.714262 14487 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.714723 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:34.708945 14702 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.715673 11397 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:34.716351 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 113602us
I20260709 13:16:34.716171 14702 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.717439 11515 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:34.706923 14739 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.719683 15129 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:34.730356 14648 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:34.731073 14648 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.720068 15217 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:34.731473 14648 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:34.732317 15217 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.720551 15079 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:34.721773 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 115997us
I20260709 13:16:34.737622 15200 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:16:34.738190 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.738014 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.718932 14702 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:34.739446 11542 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:34.722151 14863 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:16:34.741634 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.721846 15238 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:34.740499 14487 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.784950 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.786024 11394 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:34.786255 14487 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:34.728636 14739 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.743034 14257 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.729074 14976 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:16:34.789070 14976 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.790030 14976 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.791656 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.802724 11546 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:34.804844 11397 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:34.502388 14813 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.806599 11549 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:34.489813 14701 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:34.807191 14487 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:34.808641 14487 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:34.808753 14701 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.810352 15029 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:34.810801 14739 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:34.811408 14739 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:34.812407 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.812413 14257 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:34.813894 14257 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:34.815078 14257 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.816433 14257 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:34.728716 15097 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:34.818679 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:34.819651 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.820380 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.821202 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:34.821327 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.822556 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.823163 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:34.823417 11542 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 2.
I20260709 13:16:34.823544 15146 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:34.824059 15146 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:34.824378 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.824055 11395 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:34.810851 15029 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:34.835899 15029 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.836808 15146 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.837837 15029 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:34.838546 15146 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:34.843427 15099 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.844254 15176 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:34.847529 15081 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:34.848667 15159 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:34.849586 15097 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:34.853225 15166 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.853564 14813 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:34.720729 11550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.005s	sys 0.045s
W20260709 13:16:34.879827 11550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.010s	sys 0.050s
I20260709 13:16:34.879997 14813 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:34.885303 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 1949) took 1120 ms. Trace:
I20260709 13:16:34.885753 11549 rpcz_store.cc:276] 0709 13:16:33.765336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.766549 (+  1213us) service_pool.cc:224] Handling call
0709 13:16:34.885043 (+1118494us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000b1b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:34.885217 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388982528}
I20260709 13:16:34.889139 15105 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:34.891801 15154 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:34.892264 15154 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:34.891969 14648 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.893482 15154 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.896648 15154 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:34.729844 15142 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:34.867295 11542 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:34.921167 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.868501 11472 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:16:34.869050 14937 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.871126 14976 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:34.872162 15097 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:34.876724 15217 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:34.878959 15163 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:34.858603 15099 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.894831 14648 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.925539 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 595) took 2914 ms. Trace:
I20260709 13:16:34.925333 15142 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.927212 15166 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.923698 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.926152 11472 rpcz_store.cc:276] 0709 13:16:32.010865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.011047 (+   182us) service_pool.cc:224] Handling call
0709 13:16:32.013940 (+  2893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.401759 (+1387819us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54000dda20. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9890dc0 00007f8da991b1c0 00007f8da991c231 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c
0709 13:16:34.359448 (+957689us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b58009302d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da98750aa 00007f8da98f7447 00007f8da990ffe7 00007f8da99117de 00007f8da991bb49 00007f8da991c416 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4
0709 13:16:34.925081 (+565633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.925395 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550554624}
I20260709 13:16:34.946132 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1942) took 1171 ms. Trace:
I20260709 13:16:34.924386 15097 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:34.931843 15142 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.947822 11542 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:34.924674 15163 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:34.930436 14976 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:34.978410 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:34.980320 11474 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.016s	sys 0.012s
W20260709 13:16:34.983649 11474 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.016s	sys 0.012s
W20260709 13:16:34.983690 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.007s	sys 0.032s
W20260709 13:16:34.984105 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.007s	sys 0.032s
I20260709 13:16:34.984966 14739 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.986007 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.986891 11547 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:34.988303 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.989801 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.990331 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:34.990545 11547 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:34.991918 14976 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.992195 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:34.992506 15106 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:34.996506 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:34.995682 15056 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:34.994143 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:35.009840 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.010963 14685 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:35.011497 14685 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.011904 14685 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.012107 15042 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:35.012979 15042 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:34.995491 15048 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.014596 15048 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:35.015815 15033 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:16:35.015880 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1955) took 1249 ms. Trace:
I20260709 13:16:35.016372 11546 rpcz_store.cc:276] 0709 13:16:33.766698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.842506 (+ 75808us) service_pool.cc:224] Handling call
0709 13:16:33.853173 (+ 10667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.803320 (+950147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.015547 (+212227us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54000b2a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:35.015820 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049429248}
I20260709 13:16:35.017750 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.018468 11546 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:35.019507 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:35.020707 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.022725 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.011133 11547 raft_consensus.cc:2410] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:35.023357 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.024149 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.024201 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.026010 14863 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:35.026702 14863 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.027040 11547 raft_consensus.cc:2410] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:16:35.027122 14863 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.027449 11542 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:35.028024 11547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:41380
I20260709 13:16:35.028709 11547 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:35.029255 11547 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.031761 15163 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:34.973315 14937 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:35.011144 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 73589.1us
I20260709 13:16:34.986711 14739 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.119526 14685 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:35.123126 15133 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.052s
W20260709 13:16:35.123611 15133 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.006s	sys 0.052s
I20260709 13:16:35.006747 11549 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:35.032502 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 479) took 1118 ms. Trace:
I20260709 13:16:35.125078 11546 rpcz_store.cc:276] 0709 13:16:33.913817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.031476 (+1117659us) service_pool.cc:224] Handling call
0709 13:16:35.031891 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.032166 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.032192 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.032233 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:35.032282 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.032334 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.032433 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:35.122860 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.110445 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:35.111181 11548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.023s	sys 0.030s
I20260709 13:16:35.113706 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1936) took 2317 ms. Trace:
I20260709 13:16:35.035086 15097 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.115669 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 480) took 1197 ms. Trace:
I20260709 13:16:35.115520 11547 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.116691 15163 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.005880 14813 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:35.128284 11544 rpcz_store.cc:276] 0709 13:16:33.918369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.114572 (+1196203us) service_pool.cc:224] Handling call
0709 13:16:35.114966 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.115267 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.115299 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.115365 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:35.115424 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.115489 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.115601 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.146807 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.127960 11551 rpcz_store.cc:276] 0709 13:16:32.796620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.074861 (+278241us) service_pool.cc:224] Handling call
0709 13:16:33.831090 (+756229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.841398 (+ 10308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.113312 (+1271914us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b54001a6dd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:35.113536 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705670912}
I20260709 13:16:35.179471 11543 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:35.179543 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.180787 11551 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:35.181972 11551 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:35.182286 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2179) took 1336 ms. Trace:
I20260709 13:16:35.147956 11546 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:35.183480 15166 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.184747 11546 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:35.186383 11546 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:35.151392 15106 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.187296 14863 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.188868 15106 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.152032 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.189973 14863 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:35.190243 11393 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:35.155879 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.156165 15042 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.156586 14976 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.157912 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 481) took 1228 ms. Trace:
I20260709 13:16:35.160204 15048 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.160778 15097 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.161470 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.164932 15099 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.165504 15107 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:35.170343 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45175us
I20260709 13:16:35.129158 14685 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.027240 14893 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:35.174106 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:35.175135 14851 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.034149 14648 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:35.177408 11547 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:34.946702 11550 rpcz_store.cc:276] 0709 13:16:33.760923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.761141 (+   218us) service_pool.cc:224] Handling call
0709 13:16:33.761561 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.943253 (+181692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.943299 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.943982 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.880490 (+936508us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:34.880544 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.880625 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.929688 (+ 49063us) raft_consensus.cc:1685] finished
0709 13:16:34.929711 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.931913 (+  2202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1180,"raft.queue_time_us":66892,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":3971,"thread_start_us":896939,"threads_started":2,"wal-append.queue_time_us":801316}
I20260709 13:16:35.193670 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:35.194001 11548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.023s	sys 0.030s
W20260709 13:16:35.196776 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.201365 11549 rpcz_store.cc:276] 0709 13:16:33.929675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.125065 (+1195390us) service_pool.cc:224] Handling call
0709 13:16:35.157302 (+ 32237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.157543 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.157570 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.157634 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:35.157690 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.157746 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.157855 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.211318 11544 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:16:35.182785 11543 rpcz_store.cc:276] 0709 13:16:33.846257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.486129 (+639872us) service_pool.cc:224] Handling call
0709 13:16:34.486442 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.180232 (+693790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.181920 (+  1688us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540041c8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:35.182193 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472536832}
I20260709 13:16:35.214006 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:35.218618 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.011s	sys 0.020s
W20260709 13:16:35.219050 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.011s	sys 0.020s
I20260709 13:16:35.218873 11398 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:35.208278 14893 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:35.230578 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.230870 15016 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:35.230953 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:35.231606 15209 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.002s	sys 0.033s
I20260709 13:16:35.234680 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.213135 15107 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.230592 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.237444 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41380 (request call id 477) took 1305 ms. Trace:
I20260709 13:16:35.239492 11469 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:35.235293 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.269397 11473 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:35.270797 15107 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.271246 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:35.272387 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:35.273108 11551 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:35.274545 11551 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:16:35.275707 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1946) took 1511 ms. Trace:
I20260709 13:16:35.276130 11548 rpcz_store.cc:276] 0709 13:16:33.764521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.764629 (+   108us) service_pool.cc:224] Handling call
0709 13:16:33.764916 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.926681 (+161765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.926726 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.979981 (+ 53255us) log.cc:844] Serialized 26 byte log entry
0709 13:16:35.275220 (+1295239us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:35.275292 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.275408 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.275460 (+    52us) raft_consensus.cc:1685] finished
0709 13:16:35.275472 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.275640 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":70281,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":11955,"thread_start_us":1158476,"threads_started":2,"wal-append.queue_time_us":1079216}
W20260709 13:16:35.277269 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 103345us
I20260709 13:16:35.278256 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:35.279043 14851 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.222229 14937 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.283275 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:35.286505 15211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.011s	sys 0.027s
W20260709 13:16:35.286677 15209 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.002s	sys 0.033s
I20260709 13:16:35.282083 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.280323 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:35.274843 11551 pending_rounds.cc:85] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 4
I20260709 13:16:35.255540 11547 rpcz_store.cc:276] 0709 13:16:33.907618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.027794 (+1120176us) service_pool.cc:224] Handling call
0709 13:16:35.212724 (+184930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.282130 11546 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:35.289083 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53895.6us
W20260709 13:16:35.289986 11401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.007s	sys 0.018s
W20260709 13:16:35.290334 11401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.007s	sys 0.018s
W20260709 13:16:35.287014 15211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.011s	sys 0.027s
I20260709 13:16:35.292708 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:35.291028 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:35.292241 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.301354 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.302186 11397 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:35.300559 11476 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 5.
I20260709 13:16:35.288586 11551 pending_rounds.cc:107] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:16:35.308559 11551 raft_consensus.cc:2889] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:35.308660 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.312462 14851 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.313309 15187 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:35.314016 15187 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:35.315395 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.316421 15187 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:35.317013 11477 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.018s	sys 0.015s
W20260709 13:16:35.317394 11477 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.018s	sys 0.015s
I20260709 13:16:35.318282 15042 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:35.318378 11516 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.318832 15042 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.319187 14966 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:35.319306 15042 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.319693 14966 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:35.321851 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:35.322610 11396 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.291496 14685 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.296674 14487 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:35.331534 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.320227 14966 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:35.335644 11398 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:35.336874 11472 raft_consensus.cc:2393] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:35.339612 11547 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:35.280939 14648 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.287146 14813 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.343369 11476 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:35.342371 15129 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:35.348359 11545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.000s	sys 0.024s
W20260709 13:16:35.350299 11545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.001s	sys 0.025s
I20260709 13:16:35.351958 14685 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.352478 15129 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.353149 15129 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:35.354423 14685 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.353979 14893 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.336481 14487 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:35.358584 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.359484 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.348839 14851 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:35.366391 14851 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.366781 14851 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.367658 11397 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:35.367841 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 76963.2us
I20260709 13:16:35.368992 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.369922 11397 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:35.370371 11397 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:35.373365 14921 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.373919 14921 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:35.375309 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 475526us
I20260709 13:16:35.377892 15227 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:16:35.379166 15105 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:35.379310 15200 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:35.380242 14685 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:35.380439 11398 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:35.380730 14685 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:35.381297 15159 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.382231 14813 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.381803 14685 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:35.384423 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.386857 14685 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:35.381289 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 101184us
I20260709 13:16:35.390089 14921 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.390206 15153 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:35.381923 15159 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.391574 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.392537 11472 raft_consensus.cc:2410] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:35.369863 11367 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.349987 14863 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:35.352010 15207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.008s	sys 0.028s
W20260709 13:16:35.351974 15231 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.824s	user 0.005s	sys 0.001s
W20260709 13:16:35.418152 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48382.8us
I20260709 13:16:35.361876 14487 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:35.424928 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48380.7us
I20260709 13:16:35.426954 11396 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:35.429024 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:35.429311 11516 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.429925 11471 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:35.430233 15146 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:35.424942 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.430755 15146 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:35.424791 15207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.008s	sys 0.029s
I20260709 13:16:35.422119 15105 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:35.447670 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.426611 14893 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:35.423614 15187 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:16:35.434898 14701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.027s
W20260709 13:16:35.444015 15191 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.012s	sys 0.034s
I20260709 13:16:35.440706 11397 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.429016 15137 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 13:16:35.460016 14701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.005s	sys 0.027s
I20260709 13:16:35.454337 15187 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.437156 14863 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:35.474197 14979 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.011s	sys 0.045s
W20260709 13:16:35.424739 14813 consensus_queue.cc:1175] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "147201d22a694ca59f6e014f083a0892" responder_term: 9 status { last_received { term: 8 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:35.466539 15153 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:16:35.481195 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 475) took 1628 ms. Trace:
I20260709 13:16:35.481127 15146 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.484033 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.478739 15056 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:35.486621 11397 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:35.517364 14979 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.012s	sys 0.045s
I20260709 13:16:35.458643 15105 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.484112 15159 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.512053 14648 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.512522 14966 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.509864 14851 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:35.464663 15191 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.012s	sys 0.035s
W20260709 13:16:35.521791 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 48179.4us
I20260709 13:16:35.515318 15187 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:35.493897 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2222) took 1275 ms. Trace:
I20260709 13:16:35.529156 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:35.453557 14257 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:16:35.523417 11545 rpcz_store.cc:276] 0709 13:16:33.852643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.610023 (+757380us) service_pool.cc:224] Handling call
0709 13:16:34.677484 (+ 67461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.720408 (+ 42924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.720438 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.721237 (+   799us) log.cc:844] Serialized 26 byte log entry
0709 13:16:35.350688 (+629451us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:35.350723 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.350785 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.480899 (+130114us) raft_consensus.cc:1685] finished
0709 13:16:35.480931 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.481102 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40177,"raft.queue_time_us":41825,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":299,"thread_start_us":628967,"threads_started":1,"wal-append.queue_time_us":756939}
W20260709 13:16:35.551173 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 98845.2us
W20260709 13:16:35.539042 12116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.004s	sys 0.040s
I20260709 13:16:35.469148 15142 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:16:35.544497 11542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.008s	sys 0.011s
W20260709 13:16:35.544586 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.600380 11542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.008s	sys 0.011s
I20260709 13:16:35.526489 15056 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:35.606460 15056 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.607635 15056 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:35.608253 14966 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.531563 14739 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:35.552829 12116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.005s	sys 0.041s
W20260709 13:16:35.563297 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.614534 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.609794 15105 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:35.624611 11395 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.019s	sys 0.014s
W20260709 13:16:35.628528 11395 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.020s	sys 0.014s
I20260709 13:16:35.613260 14739 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.595716 15159 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:16:35.573107 15156 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.021s	sys 0.025s
W20260709 13:16:35.565186 11402 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.005s	sys 0.017s
W20260709 13:16:35.650009 11402 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.008s	sys 0.019s
I20260709 13:16:35.597556 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2224) took 1119 ms. Trace:
I20260709 13:16:35.549765 11474 rpcz_store.cc:276] 0709 13:16:34.219030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.219179 (+   149us) service_pool.cc:224] Handling call
0709 13:16:34.219504 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.219699 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:34.984155 (+764456us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.470324 (+486169us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.487990 (+ 17666us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b48003702d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:35.491326 (+  3336us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b48003702d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:35.493463 (+  2137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.493539 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:35.493598 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.493683 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.493837 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":241,"prepare.queue_time_us":799690,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":9220,"raft.run_cpu_time_us":3062,"raft.run_wall_time_us":43611,"spinlock_wait_cycles":56361472,"thread_start_us":1129434,"threads_started":2}
I20260709 13:16:35.570672 15080 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:35.547446 15154 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:35.647774 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:35.651547 11398 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:35.653831 11397 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:16:35.653281 15156 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.026s	sys 0.028s
I20260709 13:16:35.643431 14739 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:35.628566 15142 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:16:35.653330 15080 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:35.653330 14851 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.647495 15235 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:35.650552 11475 rpcz_store.cc:276] 0709 13:16:34.363210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.363612 (+   402us) service_pool.cc:224] Handling call
0709 13:16:34.363975 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:34.375071 (+ 11096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.375138 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:34.375842 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 13:16:34.980453 (+604611us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:35.445292 (+464839us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.451604 (+  6312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.463059 (+ 11455us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.466224 (+  3165us) raft_consensus.cc:1685] finished
0709 13:16:35.466244 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.471861 (+  5617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10601,"prepare.queue_time_us":30179,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":22002,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":83712,"thread_start_us":835710,"threads_started":2,"wal-append.queue_time_us":636805}
I20260709 13:16:35.677786 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 618) took 1312 ms. Trace:
I20260709 13:16:35.659480 14857 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:35.648098 14257 consensus_queue.cc:1059] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:16:35.675800 15080 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:35.700259 15236 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:35.700856 15236 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.712028 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:16:35.714131 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 324453us
I20260709 13:16:35.714066 11400 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:35.691396 14863 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:35.735390 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47709.1us
I20260709 13:16:35.678297 11477 rpcz_store.cc:276] 0709 13:16:34.365451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.388022 (+ 22571us) service_pool.cc:224] Handling call
0709 13:16:34.388406 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.388654 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:35.317786 (+929132us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.644440 (+326654us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.656290 (+ 11850us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:35.664351 (+  8061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.673538 (+  9187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.677710 (+  4172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":624494,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":124,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6915,"thread_start_us":1017032,"threads_started":2}
I20260709 13:16:35.677498 15154 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.713388 15235 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.692801 14487 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.766615 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:35.744148 15105 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.778733 15105 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.781680 12118 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:35.783970 15235 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:35.784127 15105 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:35.794499 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 220755us
I20260709 13:16:35.770552 14937 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:35.754006 15042 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.776392 15254 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:35.802145 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 207336us
I20260709 13:16:35.805138 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.821847 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.806957 12123 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:35.823607 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:35.822288 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 430291us
I20260709 13:16:35.823033 14648 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:35.847957 15211 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.848559 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:35.829531 14937 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:35.844209 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:35.848687 15211 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.804805 15129 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.860244 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.863523 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:35.865163 15235 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:35.865983 15258 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:16:35.866523 15137 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:35.867151 15137 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:35.868285 15137 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.869889 15137 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:35.870452 15259 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:16:35.870647 15097 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:35.861192 12123 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:35.876861 15211 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:35.880134 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.882813 12119 raft_consensus.cc:2368] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:35.880980 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:35.884474 12112 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.885752 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.873346 15097 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:35.892118 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.893640 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:35.895300 15277 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:35.896703 14937 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.900068 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:35.903927 15163 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.001s
I20260709 13:16:35.844691 15254 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.888553 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:35.911788 11394 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:16:35.915414 14851 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:35.915911 14851 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.916322 14851 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.911238 15154 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.918331 15097 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.919787 15097 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:35.926534 12112 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:35.926834 15042 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.913893 12117 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:35.942946 14487 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:35.943267 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:35.949802 15267 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:16:35.950562 15177 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:35.950840 15269 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:16:35.955629 15270 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:35.959506 14685 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.960367 14685 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:35.955401 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 111348us
I20260709 13:16:35.977098 14739 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:35.978315 15254 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:35.979190 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97250.1us
I20260709 13:16:35.980014 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:35.980501 15187 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:35.981359 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:35.982532 15245 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:35.983534 15245 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:35.983863 15262 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:16:35.982632 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:35.944463 14648 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:35.984051 15156 raft_consensus.cc:3055] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:16:35.982587 15253 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.271s	user 0.000s	sys 0.003s
W20260709 13:16:35.991042 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.971453 14685 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.990370 15156 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:35.982678 15187 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:35.999275 15159 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:36.025326 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.000379 15080 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.001502 15129 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:35.993816 15282 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:36.007647 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 478) took 2099 ms. Trace:
I20260709 13:16:36.028072 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.028373 11542 rpcz_store.cc:276] 0709 13:16:33.907926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.028184 (+1120258us) service_pool.cc:224] Handling call
0709 13:16:35.028584 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:35.028980 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.029006 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:35.029578 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:16:35.601071 (+571493us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:35.601127 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.601220 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.983264 (+382044us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:36.007334 (+ 24070us) raft_consensus.cc:1685] finished
0709 13:16:36.007354 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.007579 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1906,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":114,"spinlock_wait_cycles":25492864,"thread_start_us":571034,"threads_started":1,"wal-append.queue_time_us":672414}
W20260709 13:16:36.008451 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.991626 14739 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:36.010085 15252 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
W20260709 13:16:36.020153 15278 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:16:36.032011 15080 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.022887 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.034227 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:36.023499 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.006501 11365 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.035568 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:36.036245 14739 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:36.036818 14739 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:36.037614 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.037863 14739 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.039491 15187 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:36.036266 11393 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.041782 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.042596 15187 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:16:36.045406 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.047886 14994 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:36.048162 11366 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:36.049247 15159 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:36.049451 14994 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:36.049675 15159 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:36.049963 14994 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.050442 14994 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.051975 15277 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:36.049369 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.052575 15277 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.053164 11395 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:36.054585 15097 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:36.025470 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.035504 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.034873 11399 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:36.054358 15156 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.055898 15200 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:36.056308 15200 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:36.056545 15156 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.057147 15200 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.058323 15200 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:36.049980 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.058692 14994 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.054746 15277 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:36.059351 15254 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:36.050705 15159 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.060122 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:36.060019 15254 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:36.060923 15254 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:36.060014 11396 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:36.063862 15159 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:36.000844 15248 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:36.065223 15248 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:36.065639 15177 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:36.065738 11393 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:36.069855 15254 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.071897 15254 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.072468 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:36.073088 11477 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:36.074115 14994 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:36.042718 15280 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:16:36.077147 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.078178 12117 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:36.079768 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1919) took 1057 ms. Trace:
I20260709 13:16:36.080231 11394 rpcz_store.cc:276] 0709 13:16:35.021825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.024414 (+  2589us) service_pool.cc:224] Handling call
0709 13:16:35.024715 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.912206 (+887491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.079421 (+167215us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b54001270d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:36.079670 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2023935232}
I20260709 13:16:36.051901 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.081754 12118 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:36.075555 15177 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:36.082451 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.084662 15236 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:36.087508 12115 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:16:36.050036 15016 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.096395 15016 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.101030 12122 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:36.067793 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1918) took 1046 ms. Trace:
I20260709 13:16:36.101975 11400 rpcz_store.cc:276] 0709 13:16:35.021525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.021643 (+   118us) service_pool.cc:224] Handling call
0709 13:16:35.021935 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.715983 (+694048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.067510 (+351527us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54001e00d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:36.067710 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505529600}
I20260709 13:16:36.114161 15016 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.049722 11366 leader_election.cc:400] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:36.115741 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:36.116087 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 66569.2us
I20260709 13:16:36.116844 11397 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:36.117658 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:36.119817 15163 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:36.120457 15163 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:36.120568 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.121774 11397 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:16:36.123447 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 140894us
I20260709 13:16:36.124634 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:36.126340 15254 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:36.127413 15254 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:36.128883 15248 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.129105 15254 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.129997 15248 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:36.138582 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.141060 15129 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.141858 15129 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.144723 15129 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:36.144903 11401 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:36.147254 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:36.148680 15137 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:36.149096 15137 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.149405 15137 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.149962 15177 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.152992 11399 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:36.154824 15177 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:36.156487 15137 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.158676 15137 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:36.111627 15056 consensus_meta.cc:287] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 13:16:36.159953 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.160531 12114 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.161180 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.163288 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:36.164094 11475 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:16:36.165460 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.166220 12115 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:36.121372 15163 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.180476 11401 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:16:36.186784 12114 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:36.186928 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:36.054428 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.190379 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.191536 11397 raft_consensus.cc:2410] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:16:36.192263 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67765.6us
W20260709 13:16:36.196738 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.190560 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:36.208606 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.211786 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.212658 12114 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.217074 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.218143 12118 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:16:36.219261 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 164931us
W20260709 13:16:36.220361 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.221176 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.221905 12113 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.222782 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.227046 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.228767 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:36.230139 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:36.231448 11399 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:36.237615 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:36.241459 12122 raft_consensus.cc:2368] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:36.210903 14685 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:36.232028 15200 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:36.262575 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.262928 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:36.263284 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 72797.1us
I20260709 13:16:36.272789 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:36.264854 14863 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:36.231938 15235 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:36.200002 15236 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:16:36.245015 15211 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:36.240054 15137 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:36.267004 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.265992 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.285899 15177 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:36.287242 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.287853 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:36.289907 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:36.293394 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64735.7us
I20260709 13:16:36.299036 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:36.303380 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.325999 11550 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:36.294637 15129 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:36.364643 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.276043 14685 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:36.269210 15200 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:36.280468 14893 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:16:36.325646 15211 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:36.395159 15211 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.395648 15097 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:36.397677 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.304816 15137 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.399904 15137 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:36.389683 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.403906 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.404506 11550 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:36.381145 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.409228 11548 raft_consensus.cc:2368] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:36.382335 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:36.410604 14648 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:36.410964 14648 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:36.411073 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:36.411391 14648 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:36.411891 15154 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:36.412271 15154 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:36.412611 15154 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:36.419345 11544 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:36.420751 15154 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.423017 15154 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.424140 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:36.425040 15245 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:36.425428 15245 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.425757 15245 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.429292 14648 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.336562 15248 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.356106 15282 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.345805 15235 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.394057 14893 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:36.433199 15245 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.299808 15177 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:36.331383 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 133442us
W20260709 13:16:36.454105 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47126.5us
I20260709 13:16:36.467110 12113 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:36.466408 15283 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.390133 14937 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:36.390321 15200 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.158790 14851 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:36.475522 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.545642 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.508996 14893 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:36.543805 15146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.045s	sys 0.091s
I20260709 13:16:36.537173 14937 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:36.492468 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.496234 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.568693 11547 raft_consensus.cc:2368] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
I20260709 13:16:36.497897 11544 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:16:36.519289 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.569254 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:36.570039 11472 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:36.533846 15283 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.524950 15177 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:36.536721 11551 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:36.544513 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:36.565212 12112 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:36.565523 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:36.563297 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:36.478070 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:36.571835 11471 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:36.560571 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:36.572762 11549 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:36.561574 15146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.045s	sys 0.091s
I20260709 13:16:36.582146 11476 raft_consensus.cc:1240] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:36.583277 11367 leader_election.cc:400] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:36.585616 11367 leader_election.cc:403] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:16:36.585773 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 107836us
I20260709 13:16:36.576946 11516 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:36.587704 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.589627 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:36.553668 14863 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:36.509684 15282 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.595085 11477 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:16:36.597571 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:36.566555 11514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.528s (SENT) [suppressed 1 similar messages]
I20260709 13:16:36.571981 14937 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:36.599089 11402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.009s
W20260709 13:16:36.612720 11402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.009s
I20260709 13:16:36.614677 11367 leader_election.cc:400] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
I20260709 13:16:36.615226 11367 leader_election.cc:403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:36.546339 14685 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.592308 15274 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.617398 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1915) took 1596 ms. Trace:
I20260709 13:16:36.592082 15137 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:16:36.626250 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 191720us
I20260709 13:16:36.631337 11469 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:36.598825 14994 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:36.616271 15156 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:16:36.593525 15236 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:16:36.631810 11367 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:36.646291 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.623564 15099 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:36.602221 14863 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:36.646723 12117 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.008s
I20260709 13:16:36.630350 14739 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7.
W20260709 13:16:36.657444 12117 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.009s	sys 0.009s
I20260709 13:16:36.647405 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.669009 12114 raft_consensus.cc:2410] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:36.639559 15137 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:36.657006 15097 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:36.695828 15137 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.699311 14648 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:36.701407 15235 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:36.702199 15235 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:36.703075 15235 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:36.703629 15235 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:36.704679 15235 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.645641 14893 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.707944 14893 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.712023 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:36.712647 11471 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.714617 15283 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.640425 14994 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.715958 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:36.717387 11469 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:36.719022 11543 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:36.728550 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.737888 11474 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:36.756860 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:36.757831 11470 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:36.760857 14863 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:36.760936 14937 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:36.761997 15289 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:16:36.764964 11471 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:36.750597 15245 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.762444 14851 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:36.818300 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 228758us
W20260709 13:16:36.832460 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 234680us
I20260709 13:16:36.837348 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.840667 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.769932 15153 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:16:36.809352 15211 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:36.876729 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.880298 15274 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.880841 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:36.881283 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:36.881620 15163 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.882230 15163 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.892639 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:36.626858 11402 rpcz_store.cc:276] 0709 13:16:35.020562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.024928 (+  4366us) service_pool.cc:224] Handling call
0709 13:16:35.025249 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.025598 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:35.656162 (+630564us) log.cc:844] Serialized 14 byte log entry
0709 13:16:36.086310 (+430148us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b180028fc00. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:16:36.613340 (+527030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:36.613414 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:36.617090 (+  3676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.617187 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.617330 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":632356,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":11761,"spinlock_wait_cycles":12628736,"thread_start_us":1151132,"threads_started":2}
I20260709 13:16:36.821668 15080 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.902868 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:36.882735 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:36.906016 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 329105us
W20260709 13:16:36.907485 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.907737 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:36.914325 15042 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.915045 15042 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:36.905565 15211 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.917467 15042 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:36.917963 15042 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:36.918480 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:36.919081 15272 raft_consensus.cc:2764] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:16:36.919618 11397 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:36.914433 15153 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:36.920800 15153 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:36.921718 15153 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:36.922273 12113 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:36.922680 14893 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:36.918380 15042 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:37.006816 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 440261us
W20260709 13:16:37.034371 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.008880 11402 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:16:36.975589 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.910198 15097 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:37.035761 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.922936 15153 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:36.893638 15297 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:36.924001 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.038686 11401 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:36.919117 11399 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.971364 15056 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:37.027102 15274 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.924206 15156 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:36.988862 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:36.932317 15282 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:36.933571 15245 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:36.930483 15285 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:16:36.929528 15254 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:36.886374 15248 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:37.017517 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47248.2us
W20260709 13:16:36.910602 15266 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:16:36.902894 15159 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.909253 15080 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:36.926784 15293 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:16:37.051283 11549 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:16:37.079329 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 432054us
I20260709 13:16:37.088878 11542 raft_consensus.cc:2410] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:37.092204 11366 leader_election.cc:400] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
W20260709 13:16:37.099921 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.112378 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.067320 14893 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:37.039662 15097 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:37.116158 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.113612 15156 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:37.118062 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.118208 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:37.138547 11401 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:37.118767 15200 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 13:16:37.096702 15245 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.120580 15080 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.120755 15159 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.030570 15177 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.132408 11393 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:37.135316 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.094240 15163 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:37.147848 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.170790 11396 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:37.141379 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 300792us
I20260709 13:16:37.175768 15245 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.163511 15200 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.178267 15200 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.179858 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 626) took 1006 ms. Trace:
I20260709 13:16:37.140633 15097 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.180261 11475 rpcz_store.cc:276] 0709 13:16:36.173863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.174033 (+   170us) service_pool.cc:224] Handling call
0709 13:16:36.174349 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:36.174674 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.174702 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:36.175151 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:16:36.653968 (+478817us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:36.654020 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.654085 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:37.179629 (+525544us) raft_consensus.cc:1685] finished
0709 13:16:37.179654 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.179797 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":945,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":13100,"thread_start_us":477533,"threads_started":1,"wal-append.queue_time_us":567288}
I20260709 13:16:37.184617 15159 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.185907 15159 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:37.197788 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:37.200451 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:37.204922 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.205915 11393 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:37.209040 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:37.211524 11365 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:37.136368 15235 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.217244 11400 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
W20260709 13:16:37.217751 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 335176us
I20260709 13:16:37.163906 14863 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.137517 15236 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.126441 15248 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.235538 11401 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:37.224498 14937 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:37.254436 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 116726us
W20260709 13:16:37.257706 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 60007.8us
I20260709 13:16:37.258659 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:37.206713 15080 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:16:37.217115 15163 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:16:37.268932 14937 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:37.282662 15316 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:16:37.229775 14863 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.238168 15235 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.239200 15236 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.298744 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:37.286031 15282 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:37.282631 15325 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:37.282698 15327 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:37.379211 15321 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:16:37.290691 15097 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.389614 15327 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.390048 15327 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.391207 11396 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:37.379774 15333 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:16:37.290609 15156 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.383972 15341 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.395146 15282 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:37.394968 15341 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:37.384601 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92341.6us
I20260709 13:16:37.396201 15282 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.397873 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:37.385159 14685 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:37.403666 15325 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:37.405084 14863 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.385545 15248 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.406144 15248 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:37.406580 15248 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:37.408804 15177 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.386801 15042 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.388263 15056 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.263630 14893 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.302940 14937 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:37.422206 15056 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.423827 15248 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.424912 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:37.425189 15248 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:37.426121 11401 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:37.425889 15056 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.434046 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.434805 11395 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:16:37.436564 15016 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.437201 15297 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.437091 15016 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.437686 15297 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.439235 15016 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.440781 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.442829 15297 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.445034 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.461912 15097 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.483302 15235 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.421254 14937 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.494575 15236 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.496313 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:37.423137 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 167899us
I20260709 13:16:37.510972 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.518558 12113 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:37.527451 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.525242 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:37.532745 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 107903us
I20260709 13:16:37.532840 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:37.533687 15309 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:37.533915 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:37.534272 15309 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.534721 15309 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:37.525056 15324 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.538331 14739 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.538811 15324 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.540206 15325 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:37.541007 15324 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.533608 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:37.554479 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:37.557530 15325 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:16:37.556388 15177 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.558954 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.559504 15177 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.559958 15296 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.560541 15296 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.561744 15177 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.538841 14739 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.565302 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.566268 11475 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:37.556258 15328 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:37.570770 15235 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:37.570922 15302 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:16:37.572480 15320 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:37.547891 14937 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.574472 14739 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.567799 15187 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:37.575814 14937 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:37.575620 15187 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.576438 15327 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.578104 15187 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.581164 14994 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:37.581944 11477 raft_consensus.cc:1240] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:37.586483 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.587410 11470 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:37.562812 15296 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.591470 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:37.600612 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 46263.9us
I20260709 13:16:37.601917 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.603166 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.606771 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.607599 11543 raft_consensus.cc:2410] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:37.605154 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.612198 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:37.618963 11401 raft_consensus.cc:2410] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:37.615785 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.619648 11551 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:16:37.616940 11402 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:37.617481 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:37.589076 15283 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.012s	sys 0.024s
I20260709 13:16:37.622546 11397 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:16:37.623340 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:37.571189 15328 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.629050 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.629846 11549 raft_consensus.cc:2410] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:16:37.613516 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.622895 11397 pending_rounds.cc:85] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 7
I20260709 13:16:37.627539 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.627658 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.628165 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.631371 11547 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:37.628300 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.630877 11397 pending_rounds.cc:107] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:16:37.631948 11397 raft_consensus.cc:2889] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:37.586459 15327 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:37.634322 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.585913 15099 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.636066 11394 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:37.633106 15328 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:37.585645 15245 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:16:37.656123 15272 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:37.656606 15272 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:37.657408 15272 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.658339 15272 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:37.629521 15283 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.012s	sys 0.024s
I20260709 13:16:37.661053 11474 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:37.662170 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.662756 11474 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:37.664170 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:37.664989 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.666051 11474 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:37.672276 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:37.672822 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:37.673202 15296 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:37.673355 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:37.674052 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.674628 15296 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:37.675210 15296 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:37.675401 15324 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:37.675853 15324 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.676231 15324 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:37.676172 15296 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.674901 11396 raft_consensus.cc:2410] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:37.667843 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.679236 11469 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:37.666074 11516 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.674657 12085 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:37.682148 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.682715 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:37.683795 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.684721 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.685645 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.686582 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.687266 11396 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:37.688184 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:37.688972 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 488654us
I20260709 13:16:37.689173 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:37.690604 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.690325 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.691126 15146 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:37.691655 15146 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.692027 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.693949 15146 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:37.694182 15211 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:37.694723 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.696153 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.696774 11399 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:37.600260 14648 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:16:37.697829 15311 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.625595 14937 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.697865 14648 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:37.699060 14937 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.700083 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.701237 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.643375 15099 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.701969 11470 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:37.701849 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.702689 11547 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:37.667552 11476 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:37.676362 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.699402 14648 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:37.704035 11472 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11.
I20260709 13:16:37.668269 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.703970 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.704871 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.705444 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.702826 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.698563 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.707373 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.710500 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:37.708640 11543 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:37.711485 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.712116 11544 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:37.708273 11469 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.708729 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:37.715826 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49018.1us
I20260709 13:16:37.716840 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.717303 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.719177 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.719668 11397 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:37.721007 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.690330 11474 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:37.721616 11397 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:37.722503 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44213.7us
I20260709 13:16:37.705444 11542 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:37.723078 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.724184 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.727587 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.729768 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:37.730890 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.731488 11397 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:37.732650 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:37.734706 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:37.735869 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.732976 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.737288 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:37.737552 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.737888 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.728185 11394 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:37.734771 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:37.717439 11476 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:37.738739 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.737332 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.739830 11472 raft_consensus.cc:2410] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:37.737578 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.738744 11474 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:37.746938 12123 raft_consensus.cc:2393] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:37.749518 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.750075 11545 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:37.750767 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.751452 11549 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:37.707664 15235 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.711858 14966 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:37.754173 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.754647 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.755987 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.756508 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.718075 15146 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:37.722774 15211 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:37.716182 14685 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:37.701313 14937 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.762915 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.763594 11550 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:37.764078 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.764597 15042 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:37.782692 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:37.765195 14893 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:37.766222 11470 raft_consensus.cc:2393] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:37.765195 11473 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:37.767105 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.769274 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.769160 15328 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.770480 15346 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.738842 15142 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:37.771596 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.771930 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.758656 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.774166 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.774451 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:37.775573 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.733512 14851 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:37.777781 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.783955 11468 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:37.708807 15309 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.747481 14863 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:16:37.785737 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51062.6us
I20260709 13:16:37.787273 11549 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:37.832772 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:37.756951 15274 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:37.799409 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.835215 11393 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:37.807288 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 66708us
I20260709 13:16:37.811991 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:37.813431 11472 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:37.815546 11547 raft_consensus.cc:2410] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:37.814570 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.819814 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:37.813792 11398 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:37.836535 11545 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:37.837800 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.838618 11542 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:37.840368 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.825347 11476 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:37.842494 11474 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:37.826160 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51922.8us
I20260709 13:16:37.826630 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:37.777472 15146 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.833066 11396 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:37.846840 11473 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:37.813488 14994 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.738811 15314 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.835608 11551 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:37.774675 15235 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.860320 11473 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:37.847231 11544 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:37.848869 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 84866us
I20260709 13:16:37.820551 14851 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.698616 15311 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.857550 11470 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:37.863598 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.854969 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.865223 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.856166 15342 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:37.841751 15211 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:37.868786 11472 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:37.866389 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.843622 15274 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.851734 15146 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:37.880152 11470 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:37.881215 11472 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:37.887784 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.888384 11472 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:37.857084 14863 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:37.831961 15142 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.784811 14648 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.892311 11546 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.870607 14851 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:37.877868 15137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.179s	user 0.006s	sys 0.031s
I20260709 13:16:37.875808 15211 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:37.882877 15274 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:37.885463 15348 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:37.825533 15346 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.912060 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.889400 15336 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:37.861073 14994 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.898783 15329 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:37.917722 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.859804 15314 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.923142 15312 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:37.923588 15354 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:37.924872 15311 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:37.927253 15284 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:16:37.927773 14863 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:37.906656 15211 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:37.929267 15349 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:16:37.930356 15351 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:37.940294 15235 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:37.941892 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:37.943358 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:37.974169 11474 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:37.944587 15309 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.946420 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:37.905651 15137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.006s	sys 0.032s
I20260709 13:16:37.985838 12122 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:37.959926 11469 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:37.962095 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.969554 11472 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:37.920404 15336 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.918385 15348 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.995599 15336 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.995749 15348 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.958055 15329 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.998792 15341 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.999193 15329 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.961527 14648 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:37.976487 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.000749 11476 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:38.001282 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 105524us
I20260709 13:16:37.986343 15324 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:37.993992 15346 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:37.951771 11470 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:38.001816 11472 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:37.967270 15354 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.002122 11367 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:37.974215 15318 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:37.974756 15312 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:38.013593 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.013724 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.016587 11474 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:38.013787 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.014940 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.014781 11367 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:38.025789 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:38.026646 11472 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:38.027825 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:38.028534 11476 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:38.012483 15354 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:38.015654 15080 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:38.033725 15080 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.034099 15080 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:38.040691 15314 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:38.010439 15341 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:16:38.042827 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.043756 14994 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:38.017645 15318 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:38.045233 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 83233.4us
I20260709 13:16:38.045840 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:38.046056 15318 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:38.046603 15318 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:38.047114 15328 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:38.046893 15235 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:38.048347 15312 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:38.049453 15235 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:38.049862 15235 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:38.054157 15177 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:38.054616 15177 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.054777 15142 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:38.055294 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.056427 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.058125 11543 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:38.050675 15235 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:38.059744 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:38.059921 15235 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:38.059998 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:38.060344 11544 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:38.060838 11547 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:38.061328 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:38.061798 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:38.057437 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.063746 11474 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:38.064374 15361 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:38.065017 11474 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:38.064862 15361 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:38.066555 11547 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:38.059095 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:38.068334 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:38.069382 11470 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:38.067531 11543 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:38.074620 11473 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:38.075512 11474 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:38.079308 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.079998 11471 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:38.067826 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:38.085170 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.088703 11473 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11.
I20260709 13:16:38.085443 11476 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:38.086318 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.090792 11546 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:38.092520 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:38.058750 15348 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:38.092923 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:38.054942 15177 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:38.122316 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54571.1us
I20260709 13:16:38.046942 15318 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.125785 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.118644 11471 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:16:38.771242 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:38.790365 11470 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:38.450184 11547 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:38.181638 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:38.233912 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 158234us
W20260709 13:16:38.618613 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.798035 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.097419 11472 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:38.315866 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:38.695466 11399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.004s	sys 0.026s
I20260709 13:16:38.787195 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.066556 15283 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:38.788730 11367 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:38.779346 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1582) took 1681 ms. Trace:
I20260709 13:16:38.814579 11473 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:38.814792 11400 rpcz_store.cc:276] 0709 13:16:37.097879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.098076 (+   197us) service_pool.cc:224] Handling call
0709 13:16:37.124739 (+ 26663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.224487 (+ 99748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.779025 (+1554538us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b5400349754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:38.779267 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193953024}
W20260709 13:16:38.813505 11399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.006s	sys 0.040s
I20260709 13:16:38.119524 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:38.737944 15357 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.245464 15320 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.092679 15361 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:38.081849 15314 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.765s
W20260709 13:16:38.573511 15354 consensus_meta.cc:287] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.000s
I20260709 13:16:38.978545 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.228137 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:39.229622 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.323375 15324 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:38.865309 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:38.555030 15146 consensus_meta.cc:287] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.000s
I20260709 13:16:39.216427 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:38.825629 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.117625 15080 consensus_meta.cc:287] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.083s	user 0.011s	sys 0.000s
I20260709 13:16:39.093024 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:38.819762 15318 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.243062 15366 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:38.809347 15313 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:39.264904 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.844044 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:38.892234 11545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.010s	sys 0.026s
W20260709 13:16:38.985955 15371 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:39.227612 11474 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:16:39.231000 11472 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
W20260709 13:16:39.013185 11443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.011s	sys 0.028s
I20260709 13:16:38.987068 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.234108 15314 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:39.233587 15354 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:38.813488 15283 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:39.230269 15320 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.230536 15357 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.243472 15348 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:38.818311 15099 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.116s	user 0.012s	sys 0.000s
W20260709 13:16:38.868312 15336 consensus_meta.cc:287] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.002s
I20260709 13:16:39.259186 11473 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:39.088815 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:39.290896 15235 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 1.230s	user 0.007s	sys 0.003s
I20260709 13:16:39.301968 11471 raft_consensus.cc:2410] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:39.304879 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:39.305675 11474 raft_consensus.cc:2376] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:39.306808 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.307682 11474 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:39.277637 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1603) took 1203 ms. Trace:
I20260709 13:16:39.308763 11394 rpcz_store.cc:276] 0709 13:16:38.074160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.807410 (+733250us) service_pool.cc:224] Handling call
0709 13:16:39.277574 (+470164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:39.267429 11545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.010s	sys 0.036s
I20260709 13:16:39.278053 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.278239 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.310037 11470 raft_consensus.cc:2376] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:39.311164 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:39.311846 11470 raft_consensus.cc:2376] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
W20260709 13:16:39.277318 11443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.014s	sys 0.040s
W20260709 13:16:39.314806 11395 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.006s	sys 0.027s
W20260709 13:16:39.315346 11395 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.007s	sys 0.027s
I20260709 13:16:39.315373 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.315810 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.228658 15360 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.280027 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.264494 15366 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.263859 15146 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:39.288026 12084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.011s	sys 0.007s
I20260709 13:16:39.289551 15357 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.290364 15336 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.316419 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1600) took 1243 ms. Trace:
W20260709 13:16:39.319321 12084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.011s	sys 0.007s
I20260709 13:16:39.319350 11476 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:39.319536 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.025s
I20260709 13:16:39.322820 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:39.342971 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.010s	sys 0.025s
I20260709 13:16:39.343292 11470 raft_consensus.cc:2376] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
W20260709 13:16:39.343461 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 554788us
W20260709 13:16:39.320427 15309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.014s	sys 0.050s
I20260709 13:16:39.280896 15314 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.279645 15373 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:39.345034 15309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.014s	sys 0.050s
I20260709 13:16:39.320194 15366 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:39.349766 12123 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:39.350123 15297 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.350623 15146 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.355183 11549 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:39.355998 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.403326 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.321888 11514 connection.cc:256] RPC call timeout handler was delayed by -1.1362400699999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
I20260709 13:16:39.356328 14863 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.356753 15137 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.357211 15363 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:39.354892 14851 consensus_meta.cc:287] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.441s	user 0.010s	sys 0.000s
I20260709 13:16:39.266291 15080 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:39.358054 15211 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.370s	user 0.007s	sys 0.003s
I20260709 13:16:39.399274 11367 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:39.309546 15099 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:39.408648 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.340796 11395 rpcz_store.cc:276] 0709 13:16:38.073275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.081501 (+  8226us) service_pool.cc:224] Handling call
0709 13:16:38.081781 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.081948 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:38.501025 (+419077us) log.cc:844] Serialized 14 byte log entry
0709 13:16:39.315987 (+814962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:39.316054 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:39.316130 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.316216 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.316358 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1010752,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1746,"thread_start_us":1068416,"threads_started":2}
W20260709 13:16:39.358280 11366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.006s	sys 0.016s
I20260709 13:16:39.412796 14863 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.412878 15137 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.413009 15363 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.410224 15355 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.408682 15297 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.358552 15314 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:39.272218 15313 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:39.436173 11366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.011s	sys 0.020s
I20260709 13:16:39.436009 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:39.450008 11470 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:39.361102 15283 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.360910 15056 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.413844 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:39.416043 15348 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:39.325637 15360 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.415885 15080 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.413182 15383 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.437501 15146 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:39.494590 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 80850.3us
I20260709 13:16:39.417788 14851 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.507726 15381 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:39.508163 14851 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.508383 15381 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.512202 15363 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:39.512555 12113 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:39.512810 15363 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.513367 15363 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.514118 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 717) took 1794 ms. Trace:
I20260709 13:16:39.516352 15297 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.517171 15383 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.519366 15320 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.519296 15146 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:39.521253 15177 consensus_meta.cc:287] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.000s
I20260709 13:16:39.521521 15137 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:39.521662 15177 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.522101 15146 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:39.400949 15373 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:39.523840 15274 consensus_meta.cc:287] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.562s	user 0.012s	sys 0.000s
I20260709 13:16:39.520107 15320 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.514616 12113 rpcz_store.cc:276] 0709 13:16:37.719474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.719571 (+    97us) service_pool.cc:224] Handling call
0709 13:16:37.738463 (+ 18892us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.513107 (+1774644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.513781 (+   674us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540043d854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:39.514018 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4075038464}
I20260709 13:16:39.470630 15361 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:39.510807 15381 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.438378 15211 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.496935 15309 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:16:39.524233 15274 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.601256 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:39.543329 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1991) took 1777 ms. Trace:
I20260709 13:16:39.662024 11475 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:39.662665 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:39.661170 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 163250us
I20260709 13:16:39.663980 11470 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:39.631042 12083 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:39.492379 15348 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.630712 14966 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.543358 15320 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.663448 15146 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:39.689879 15368 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 13:16:39.691205 15381 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.696480 15381 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.696906 15320 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:39.691744 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 432984us
I20260709 13:16:39.693408 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:39.699638 11548 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:39.667707 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 263413us
W20260709 13:16:39.700428 11514 connection.cc:256] RPC call timeout handler was delayed by -1.0957400699999198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:16:39.695632 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 64707.1us
I20260709 13:16:39.697460 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1989) took 1932 ms. Trace:
I20260709 13:16:39.648377 15375 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:39.662302 11551 rpcz_store.cc:276] 0709 13:16:37.765687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.857102 (+ 91415us) service_pool.cc:224] Handling call
0709 13:16:37.867998 (+ 10896us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:38.071370 (+203372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.071400 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:38.222894 (+151494us) log.cc:844] Serialized 52 byte log entry
0709 13:16:38.852830 (+629936us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:38.852903 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.853002 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.543116 (+690114us) raft_consensus.cc:1685] finished
0709 13:16:39.543133 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.543266 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65686,"raft.queue_time_us":86958,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3751,"thread_start_us":557748,"threads_started":2,"wal-append.queue_time_us":948573}
I20260709 13:16:39.494046 15379 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.711582 11548 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:39.542402 15080 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.701678 11550 rpcz_store.cc:276] 0709 13:16:37.765182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.848202 (+ 83020us) service_pool.cc:224] Handling call
0709 13:16:37.848602 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.911004 (+ 62402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.911050 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.946032 (+ 34982us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.107278 (+161246us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:38.107658 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 13:16:38.825035 (+717377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.825133 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.697236 (+872103us) raft_consensus.cc:1685] finished
0709 13:16:39.697255 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.697386 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":874980,"prepare.queue_time_us":10108,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":817557,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":17488,"spinlock_wait_cycles":13568,"thread_start_us":130693,"threads_started":2,"wal-append.queue_time_us":1000478}
I20260709 13:16:39.526012 15355 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:39.596063 15329 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.596s	user 0.014s	sys 0.000s
I20260709 13:16:39.772815 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:39.795802 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.710703 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:39.713728 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:39.646302 15359 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:39.862943 11550 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:39.863245 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:39.864193 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:39.859119 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 604) took 2023 ms. Trace:
I20260709 13:16:39.864666 11543 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:39.864972 12118 rpcz_store.cc:276] 0709 13:16:37.835508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.835641 (+   133us) service_pool.cc:224] Handling call
0709 13:16:37.852097 (+ 16456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.871517 (+ 19420us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:37.871565 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:37.871623 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:37.871673 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.858822 (+1987149us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5400745254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:39.858871 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.859050 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44014464}
I20260709 13:16:39.698778 15363 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.960211 15363 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.960278 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:39.960896 15363 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.706964 15388 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.963366 15388 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.965670 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:39.966544 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:39.830358 15361 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.967814 15363 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:39.968180 14648 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.957s	user 0.006s	sys 0.006s
I20260709 13:16:39.968819 14648 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.969085 15361 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:39.813288 15354 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.969445 14648 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.860982 15379 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:39.695502 15393 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:40.000440 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.001017 15393 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.001277 12119 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:40.001566 15393 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:39.707489 15385 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:40.005558 15385 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.008522 15385 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.009276 15385 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:39.628942 15373 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.628165 15360 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:39.950500 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.965826 15388 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:39.966010 15318 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.000s
I20260709 13:16:39.701162 15348 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:39.494699 15056 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:39.706645 15365 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:16:39.876817 15399 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.252s
W20260709 13:16:40.115733 15374 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.669s	user 0.000s	sys 0.000s
I20260709 13:16:39.707335 15177 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.980626 11475 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:39.808588 14851 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:39.710109 15320 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:39.966498 12118 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:39.999329 15379 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.999590 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:39.874588 15329 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.812759 15381 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.862176 15359 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:40.145411 15359 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.146018 15211 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.146512 15359 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.146850 15274 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:39.647384 15364 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.496s
I20260709 13:16:40.062816 12116 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:40.121425 15383 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.811154 15099 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:39.543551 15297 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:40.152313 15297 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.154628 15297 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.068352 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:39.807888 14863 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:39.814424 15336 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.060909 15391 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:40.157271 15391 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:39.973023 15377 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.695829 15146 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.134341 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.158905 11542 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:40.137969 14851 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.164249 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 499) took 2440 ms. Trace:
I20260709 13:16:40.164641 11546 rpcz_store.cc:276] 0709 13:16:37.724003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.747951 (+ 23948us) service_pool.cc:224] Handling call
0709 13:16:37.763171 (+ 15220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.091332 (+328161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.163996 (+2072664us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001ec8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.164171 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296684288}
I20260709 13:16:40.165540 15329 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.165973 14851 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.166113 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 540) took 2439 ms. Trace:
I20260709 13:16:40.166729 12123 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:40.167691 15329 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.168203 15329 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:40.163082 15384 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.666s	user 0.000s	sys 0.000s
I20260709 13:16:40.171151 15146 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.163245 15391 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.163774 15377 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.172730 15329 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.173245 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1604) took 2098 ms. Trace:
I20260709 13:16:40.173719 11393 rpcz_store.cc:276] 0709 13:16:38.074474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.824936 (+750462us) service_pool.cc:224] Handling call
0709 13:16:38.864157 (+ 39221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.864404 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.864432 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.216504 (+352072us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.691450 (+474946us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:39.691492 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.691569 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.172953 (+481384us) raft_consensus.cc:1685] finished
0709 13:16:40.172982 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.173173 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":419135,"threads_started":1,"wal-append.queue_time_us":836588}
I20260709 13:16:40.175371 11472 raft_consensus.cc:2410] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:16:40.175467 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.176397 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1548) took 1066 ms. Trace:
I20260709 13:16:40.176748 11472 rpcz_store.cc:276] 0709 13:16:39.109813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.315605 (+205792us) service_pool.cc:224] Handling call
0709 13:16:40.176228 (+860623us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540039a8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:40.176345 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974886656}
I20260709 13:16:40.177074 15313 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:40.177750 11365 connection.cc:256] RPC call timeout handler was delayed by -1.5436168899998393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 13:16:40.178119 11365 connection.cc:256] RPC call timeout handler was delayed by -1.5004168899997694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 13:16:40.178148 15378 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.684s	user 0.000s	sys 0.001s
W20260709 13:16:40.178504 11365 connection.cc:256] RPC call timeout handler was delayed by -1.4950168899997607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:16:40.178920 11365 connection.cc:256] RPC call timeout handler was delayed by -1.4500168899999153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 13:16:40.179071 15377 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:40.179322 11365 connection.cc:256] RPC call timeout handler was delayed by -1.3987168899998323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:16:40.179781 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 725) took 2134 ms. Trace:
I20260709 13:16:40.180272 12122 rpcz_store.cc:276] 0709 13:16:38.045269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.076810 (+ 31541us) service_pool.cc:224] Handling call
0709 13:16:38.077125 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.077447 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.077476 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.092332 (+ 14856us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.862973 (+770641us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:38.863053 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.863151 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.179496 (+1316345us) raft_consensus.cc:1685] finished
0709 13:16:40.179524 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.179689 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115264,"raft.queue_time_us":209,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11627,"thread_start_us":459901,"threads_started":1,"wal-append.queue_time_us":1044841}
W20260709 13:16:40.179839 11365 connection.cc:256] RPC call timeout handler was delayed by -1.3517134179999175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 13:16:40.182327 15329 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:39.666287 15283 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.182861 15329 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.139895 15320 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.185317 15329 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:40.185992 11365 connection.cc:256] RPC call timeout handler was delayed by -1.2808168899998691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 13:16:40.186839 11365 connection.cc:256] RPC call timeout handler was delayed by -1.1057902789998479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:16:40.187309 11365 connection.cc:256] RPC call timeout handler was delayed by -1.0331218299998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:16:40.186403 15329 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.166479 11399 rpcz_store.cc:276] 0709 13:16:37.727157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.727269 (+   112us) service_pool.cc:224] Handling call
0709 13:16:37.755674 (+ 28405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.984401 (+228727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.984449 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.989705 (+  5256us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.817341 (+827636us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:38.817392 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.165731 (+1348339us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540019a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.165777 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.165824 (+    47us) raft_consensus.cc:1685] finished
0709 13:16:40.165838 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.166037 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3637,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":11392,"spinlock_wait_cycles":524925824,"thread_start_us":792608,"threads_started":1,"wal-append.queue_time_us":1259093}
I20260709 13:16:40.188479 15329 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.189342 15099 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:40.190349 15056 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:40.190171 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.193404 15099 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:40.193830 15099 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.194681 15099 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.195880 15099 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:40.198349 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1990) took 2432 ms. Trace:
I20260709 13:16:40.198752 11545 rpcz_store.cc:276] 0709 13:16:37.765423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.850026 (+ 84603us) service_pool.cc:224] Handling call
0709 13:16:37.850485 (+   459us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:38.071647 (+221162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.071675 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:38.086144 (+ 14469us) log.cc:844] Serialized 78 byte log entry
0709 13:16:39.316969 (+1230825us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:39.712567 (+395598us) log.cc:844] Serialized 14 byte log entry
0709 13:16:39.712750 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.712816 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.198094 (+485278us) raft_consensus.cc:1685] finished
0709 13:16:40.198114 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.198286 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47954,"prepare.queue_time_us":823517,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":246378,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":13755,"thread_start_us":1547540,"threads_started":3,"wal-append.queue_time_us":1087105}
I20260709 13:16:40.119158 15318 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.200958 15329 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.206120 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 1986) took 2441 ms. Trace:
I20260709 13:16:40.207356 14648 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.209625 15355 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.211402 15318 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:40.211191 11549 rpcz_store.cc:276] 0709 13:16:37.764380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.832949 (+ 68569us) service_pool.cc:224] Handling call
0709 13:16:37.833201 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:39.402688 (+1569487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.205830 (+803142us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400405454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.206042 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3499815168}
I20260709 13:16:40.099311 15373 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:40.214789 15401 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:40.217490 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.218026 15329 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:40.221896 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.230106 11469 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:16:40.230312 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 162045us
I20260709 13:16:40.245200 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.230574 15364 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:40.247077 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:40.236131 11514 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.528s (SENT)
I20260709 13:16:40.249543 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.240262 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:40.250742 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.020s	sys 0.050s
I20260709 13:16:39.695182 14966 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.247278 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:40.251106 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.020s	sys 0.051s
W20260709 13:16:40.251470 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 540199us
I20260709 13:16:40.244860 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:40.254225 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.254930 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.256215 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.256763 12123 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:40.255589 11365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.255553 11469 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:39.705184 15375 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.258848 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.258911 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.260012 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.261128 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.261576 11469 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:40.252998 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.266072 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.237382 15401 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.268536 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.276779 12116 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:40.243319 15099 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:40.277878 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:40.278370 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.279069 11472 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:40.279634 11514 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.623s (SENT)
I20260709 13:16:40.280090 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.280150 11469 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:16:40.274816 11547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.009s	sys 0.009s
I20260709 13:16:40.270025 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.224682 15385 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:16:40.285754 11547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.010s	sys 0.009s
I20260709 13:16:40.248503 15364 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:40.285175 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:40.285573 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.259362 15359 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:40.262162 15341 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:40.262478 15399 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:40.262857 15177 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.274900 15318 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:40.274467 14648 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:40.279809 15401 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.268493 15391 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:16:40.304262 14863 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:40.322599 15366 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.914s	user 0.008s	sys 0.002s
I20260709 13:16:40.303395 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.324420 11396 raft_consensus.cc:606] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.324640 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:40.305986 15394 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:40.306672 15392 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:40.331872 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.332948 12116 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:40.307662 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.334199 12112 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 5: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:40.308828 11472 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:16:40.309101 11514 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.716s (SENT)
I20260709 13:16:40.268282 15380 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:16:40.293529 15364 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:40.315217 15097 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:40.324985 11396 raft_consensus.cc:614] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:40.243594 15373 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.305622 15400 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:40.343850 15402 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:40.344441 15402 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.344806 15402 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:40.327700 14648 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:40.327921 15394 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.351210 15394 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:40.330544 15366 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.357887 15366 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.358566 15320 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.269013 14851 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:16:40.337191 15283 consensus_queue.cc:1059] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:16:40.304800 15341 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.364377 15341 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.328856 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.287005 15355 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:40.366446 15355 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:40.366904 15355 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:40.306864 15403 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:40.370664 15403 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.371228 15403 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.335270 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.293399 15080 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.295807 15359 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:40.361007 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.326337 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.374424 12083 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:40.375171 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:40.387921 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61707us
I20260709 13:16:40.387777 15320 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.387945 11476 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
W20260709 13:16:40.366885 15367 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.754s	user 0.009s	sys 0.013s
I20260709 13:16:40.398943 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:40.399226 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 70477.1us
I20260709 13:16:40.400736 11470 raft_consensus.cc:2410] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:16:40.400712 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.401494 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:40.394124 15314 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.897s	user 0.005s	sys 0.005s
W20260709 13:16:40.405752 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 96589.3us
I20260709 13:16:40.429622 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.432104 11365 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.385134 15359 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:40.416033 15283 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:40.397065 15318 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:40.420710 15099 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:40.422936 15404 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:40.423861 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 634) took 2947 ms. Trace:
W20260709 13:16:40.425402 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.490844 11477 rpcz_store.cc:276] 0709 13:16:37.476472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.476678 (+   206us) service_pool.cc:224] Handling call
0709 13:16:37.521059 (+ 44381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.582429 (+ 61370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.423623 (+2841194us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b540038a4d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.423797 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139495296}
W20260709 13:16:40.432755 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.322845 15393 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.420125 15373 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:40.378280 15399 consensus_queue.cc:1059] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:16:40.389256 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:40.395462 15348 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:40.491773 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.502677 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.015s	sys 0.038s
W20260709 13:16:40.502921 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.503193 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.015s	sys 0.038s
W20260709 13:16:40.492282 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.492972 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.443843 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:40.405839 15248 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:40.388854 15080 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.508279 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:40.499876 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 110697us
I20260709 13:16:40.505355 11470 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:40.509485 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1598) took 2646 ms. Trace:
W20260709 13:16:40.507596 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63838.5us
I20260709 13:16:40.495060 15380 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.490000 15342 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:40.515606 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.508973 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.513294 12113 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:40.491408 15099 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.513718 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.519786 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.520334 11543 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:40.520462 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.521224 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.494639 15283 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.516464 11550 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:40.522759 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.524544 11476 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:40.436082 15314 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.437734 15366 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:16:40.504725 15320 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:40.506132 15248 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.513535 14994 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:40.509886 11398 rpcz_store.cc:276] 0709 13:16:37.862797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.915676 (+ 52879us) service_pool.cc:224] Handling call
0709 13:16:37.915999 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.919862 (+  3863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.919890 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.920362 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.077632 (+157270us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:38.077668 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.077731 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.365950 (+2288219us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:40.504250 (+138300us) raft_consensus.cc:1685] finished
0709 13:16:40.504269 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.509418 (+  5149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3497,"raft.queue_time_us":4233,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":15937,"spinlock_wait_cycles":33921792,"thread_start_us":93277,"threads_started":1,"wal-append.queue_time_us":403557}
I20260709 13:16:40.488152 15097 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:16:40.536566 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.537317 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:40.529853 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.530514 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.531168 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.539409 11544 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:40.540596 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.525930 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.542033 11542 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:40.527524 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:40.534032 11514 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.872s (SENT)
I20260709 13:16:40.513366 15398 raft_consensus.cc:3055] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:40.508831 15404 raft_consensus.cc:697] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:40.522934 15312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.920s	user 0.003s	sys 0.024s
I20260709 13:16:40.542968 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:40.546245 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.546824 11548 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:40.543344 11543 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:40.543362 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.543339 11550 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:40.521571 15099 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.548319 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.548934 11550 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:40.548949 11442 connection.cc:256] RPC call timeout handler was delayed by -1.5531887250001546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:16:40.549856 11442 connection.cc:256] RPC call timeout handler was delayed by -1.4694887250000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 13:16:40.551162 11397 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:40.555020 11514 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.888s (SENT)
I20260709 13:16:40.555649 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:40.550352 11442 connection.cc:256] RPC call timeout handler was delayed by -1.4181887250001637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 13:16:40.556910 11442 connection.cc:256] RPC call timeout handler was delayed by -1.4100887250001506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:16:40.557252 11442 connection.cc:256] RPC call timeout handler was delayed by -1.3857887250001113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 13:16:40.557729 11442 connection.cc:256] RPC call timeout handler was delayed by -1.3272887250000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:16:40.558140 11442 connection.cc:256] RPC call timeout handler was delayed by -1.3173887250000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 13:16:40.558539 11442 connection.cc:256] RPC call timeout handler was delayed by -1.2651887250001437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 13:16:40.558923 11442 connection.cc:256] RPC call timeout handler was delayed by -1.18868872500002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:16:40.559321 11442 connection.cc:256] RPC call timeout handler was delayed by -1.1850887250000142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:16:40.559676 11442 connection.cc:256] RPC call timeout handler was delayed by -1.1796887250000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:16:40.560047 11442 connection.cc:256] RPC call timeout handler was delayed by -1.1112887250001222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 13:16:40.561225 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0896887250000873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 13:16:40.561596 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0878887250000844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
I20260709 13:16:40.557005 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.562192 11542 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:40.563270 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0572887250000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 13:16:40.531086 15320 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:40.564821 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0563887250000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 13:16:40.550374 15312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.999s	user 0.004s	sys 0.024s
I20260709 13:16:40.556473 14648 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:40.566977 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0473887250000189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 13:16:40.573338 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:40.573639 11514 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.921s (SENT)
I20260709 13:16:40.574149 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.576226 15080 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.570456 15320 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.545253 15398 raft_consensus.cc:740] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.557982 15318 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.556630 15400 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:40.604440 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 208731us
W20260709 13:16:40.605487 11514 leader_election.cc:336] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.930s (SENT)
I20260709 13:16:40.531774 15380 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:16:40.564396 15385 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.566557 15146 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.603891 15366 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:40.630182 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 111424us
I20260709 13:16:40.633119 15416 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.652238 15400 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.603822 15248 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.655071 15283 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.659025 15400 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.660871 15314 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.662657 15404 consensus_queue.cc:237] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.666136 15348 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.666412 15393 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.666297 15320 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.637040 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.637571 11367 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.639734 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.603235 14648 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.651345 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.647742 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:40.652779 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79524us
I20260709 13:16:40.624079 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.616995 14994 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.646551 15355 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.619915 15388 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:40.667845 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.689260 11477 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:40.669505 15392 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:40.670132 15410 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:16:40.633497 15423 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.636690 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:40.704192 11468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43630 (request call id 2264) took 2986 ms (client timeout 2092 ms). Trace:
I20260709 13:16:40.627151 15342 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.678561 11544 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:40.704648 11468 rpcz_store.cc:269] 0709 13:16:37.718052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.732722 (+ 14670us) service_pool.cc:224] Handling call
0709 13:16:40.703998 (+2971276us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b5400059954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:40.704105 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108184832}
I20260709 13:16:40.705616 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.706326 11544 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.705505 15342 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.704182 15423 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.710034 15423 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.681241 15416 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.711639 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.712284 11477 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.713523 15416 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.691571 15364 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.720379 11547 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:40.720568 14648 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.721352 15364 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.721951 15401 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.723996 15401 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.687614 15359 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.694711 15146 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:40.686825 15398 consensus_queue.cc:260] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.729207 15398 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:40.729422 15359 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.680858 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.681074 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.648768 15341 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.682489 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 720) took 2962 ms. Trace:
I20260709 13:16:40.653956 15385 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.684995 15373 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:40.685433 15274 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:40.683616 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:40.653033 15366 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.687140 15402 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:40.691165 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85719.4us
W20260709 13:16:40.702382 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.704463 11545 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:40.754820 11514 leader_election.cc:336] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.964s (SENT)
I20260709 13:16:40.739370 11544 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
W20260709 13:16:40.755806 11514 leader_election.cc:336] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.990s (SENT)
W20260709 13:16:40.756577 11514 leader_election.cc:336] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.990s (SENT)
I20260709 13:16:40.739533 11551 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:40.740123 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59040.1us
I20260709 13:16:40.759210 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.738161 12115 rpcz_store.cc:276] 0709 13:16:37.720301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.724874 (+  4573us) service_pool.cc:224] Handling call
0709 13:16:37.755293 (+ 30419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.785119 (+ 29826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.785148 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:37.785201 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:37.785231 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.682251 (+2897020us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b54006f04d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.682298 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.682420 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68131072}
I20260709 13:16:40.743083 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.743072 11477 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
W20260709 13:16:40.752526 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 68981.1us
W20260709 13:16:40.761788 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 124301us
I20260709 13:16:40.763625 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:40.765880 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.766695 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.767496 11477 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:40.704180 15380 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:40.771992 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:40.741744 14648 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:40.772045 11514 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.051s (SENT)
I20260709 13:16:40.774382 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.773358 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:40.742738 15373 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.779078 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.780809 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:40.781934 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 110139us
I20260709 13:16:40.781545 12123 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:40.784893 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.785972 11477 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.786854 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.761737 15324 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:40.711644 15394 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:40.789927 11514 leader_election.cc:336] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.076s (SENT)
I20260709 13:16:40.791217 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.788872 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:40.794478 15382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.016s	sys 0.000s
I20260709 13:16:40.795197 15418 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.802796 14966 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:40.797538 15382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.016s	sys 0.000s
I20260709 13:16:40.801853 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.806622 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.808440 15385 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.788213 15380 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.814091 15380 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.820225 15402 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.823076 15403 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.826861 15388 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:40.742856 15341 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.830402 11477 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:40.788872 15373 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:40.781437 15392 raft_consensus.cc:2764] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:16:40.783593 15401 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.784308 15422 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:40.838949 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50136.2us
I20260709 13:16:40.839538 15411 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:40.839548 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 53843.5us
W20260709 13:16:40.838836 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48258.3us
I20260709 13:16:40.831329 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.849057 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.794198 15359 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:40.793510 14648 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.851210 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:40.839959 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43984.8us
I20260709 13:16:40.790598 15324 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:40.855537 11514 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.170s (SENT)
I20260709 13:16:40.848276 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:40.845224 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:40.836386 15309 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:40.857560 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.858374 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:40.853618 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:40.856354 15400 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:40.863564 11367 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.783736 14994 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.867964 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.872182 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.851780 15411 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:40.877480 11514 leader_election.cc:336] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.208s (SENT)
W20260709 13:16:40.879073 11514 leader_election.cc:336] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 2.332s (SENT)
I20260709 13:16:40.879528 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.880141 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.867333 15405 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:40.877512 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.862479 15324 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.881275 15392 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:40.856839 15284 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.872968 15422 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.888607 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:40.888690 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.887439 11395 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:40.863701 15401 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:40.888795 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.889973 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.867527 15342 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:40.869012 15402 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:40.864962 15403 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.827636 15366 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.870347 15385 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:40.885190 15417 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.895900 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:40.896317 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.877640 15344 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.885984 15283 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.881783 15411 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.897095 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.901450 15354 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:40.905946 15099 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.912848 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.947261 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.950546 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.917986 11545 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:40.885368 15405 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:40.894140 15388 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:40.932647 11402 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:16:40.892745 15401 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:40.916114 15355 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:40.938653 15344 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.953186 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.956581 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.935595 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.896899 15318 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:40.958429 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:40.958957 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64281.4us
W20260709 13:16:40.962292 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 90221.4us
I20260709 13:16:40.963137 11545 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:40.967154 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:40.957643 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.967173 11401 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:40.967919 11395 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:40.969026 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:40.969107 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:40.969517 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:40.969758 11399 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:40.970947 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:40.971598 11542 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:40.969843 11402 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:40.976460 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.959556 15366 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:40.981110 15403 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:40.981619 15354 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:40.981626 15422 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:40.971626 15385 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:16:40.961930 15359 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:40.926223 15417 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.986060 15405 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:40.930449 15283 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:40.987725 15394 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:40.991988 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.031270 11470 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:40.993764 11397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1602) took 2919 ms (client timeout 2679 ms). Trace:
W20260709 13:16:41.032487 11397 rpcz_store.cc:269] 0709 13:16:38.073928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.824008 (+750080us) service_pool.cc:224] Handling call
0709 13:16:40.993499 (+2169491us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400199954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:40.993659 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2998729600}
I20260709 13:16:41.034076 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:40.995446 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.035559 11472 raft_consensus.cc:2410] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:16:41.036403 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.036823 11472 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:40.996687 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:40.999209 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.037580 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.037882 11476 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.038010 11468 raft_consensus.cc:3060] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.038057 11472 raft_consensus.cc:2376] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10. Re-sending same reply.
I20260709 13:16:41.039062 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.040087 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.041015 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.041898 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.000397 15099 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.042816 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.000459 11543 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:41.044039 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.044534 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.044991 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.001016 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.001248 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:41.002014 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.002923 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.046135 11394 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:41.046267 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45091us
I20260709 13:16:41.046579 11548 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:41.047034 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.047181 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.048022 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.048949 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.049222 11548 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.049852 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.007345 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.050225 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.050627 11401 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:41.046307 11395 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:41.051237 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.052033 15099 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:41.052271 15423 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:41.052356 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.051542 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:40.959365 15401 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:41.010723 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.012460 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:41.013607 11399 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
W20260709 13:16:41.054069 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.051488 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:41.016718 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 110434us
I20260709 13:16:40.976022 15402 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.029838 11545 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:41.054695 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.054761 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.004776 15341 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:41.049892 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.013607 11542 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:41.055476 11397 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:41.043453 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.031368 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.056136 11399 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:41.019526 15354 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:41.081857 11476 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:41.082273 11402 raft_consensus.cc:2393] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:41.082715 11470 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:41.083045 11394 raft_consensus.cc:2410] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:41.059186 11468 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:41.018081 15355 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:16:41.074440 11472 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:41.076509 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.061127 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.062470 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.083639 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:41.052492 15099 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:41.086383 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.086555 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:41.085750 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54481.1us
I20260709 13:16:41.088246 11476 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.088538 11365 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:41.088882 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.089648 11399 raft_consensus.cc:2410] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:16:41.090119 15380 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:41.090662 15380 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.090867 15099 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:41.091909 15348 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:41.093042 11400 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:41.103297 11472 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:41.096355 15438 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:41.097781 15395 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:41.097967 15417 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:41.100063 11468 raft_consensus.cc:1240] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:41.109251 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.088394 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.104746 11397 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:41.059442 15403 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:16:41.105928 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:41.113818 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.112300 15418 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:41.087802 15309 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:41.111833 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49418.3us
I20260709 13:16:41.115626 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.111768 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.103747 15177 raft_consensus.cc:2804] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:41.125968 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.098650 15324 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.106786 15395 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:41.130138 15436 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.124092 14994 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:41.133610 15436 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.134028 15436 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:41.135468 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 461118us
I20260709 13:16:41.136507 12083 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.138733 14994 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.117130 15348 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.140014 14994 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.141029 15348 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.130349 15422 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:41.143378 15437 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:16:41.119067 15318 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:41.125645 15388 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:16:41.144901 15313 consensus_queue.cc:1059] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:41.145933 15393 raft_consensus.cc:3055] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:41.146446 15393 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:41.146788 15283 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:41.144913 15297 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.147482 15393 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.149142 15393 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.149737 15373 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.077082 15411 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.152488 11395 raft_consensus.cc:2368] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11.
I20260709 13:16:41.163687 15411 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:41.163395 15380 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.172683 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.174057 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.181988 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.116484 15418 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.119525 15309 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.184226 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.188614 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.189787 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.183514 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.180871 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:41.185372 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.185617 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.193161 11516 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.193933 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.195139 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.196291 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.195575 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.239113 15129 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:41.245995 15296 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:41.246438 15129 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.246658 15296 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.186375 15418 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:41.191768 15153 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.245735 15394 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 13:16:41.246678 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51218.3us
I20260709 13:16:41.158399 15401 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.249516 11401 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:41.191547 15388 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:16:41.251899 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:41.251950 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56915.8us
I20260709 13:16:41.198585 15318 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:41.200439 15159 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.246974 15129 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:41.167784 15177 raft_consensus.cc:697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:41.255491 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 74728.3us
I20260709 13:16:41.186563 15016 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:41.162175 15354 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:41.245321 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47967.5us
I20260709 13:16:41.270539 15420 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:41.271358 15373 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.271636 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.272383 11468 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:41.273310 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.274453 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.252300 15364 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.276052 15364 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.276489 15364 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.258656 15311 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:41.280004 15311 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:41.280170 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.254873 15417 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:41.280838 15417 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.281240 15417 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.280951 15311 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.251082 15416 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:16:41.247634 15296 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.283371 15146 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:41.260669 15401 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.187268 15283 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:41.280972 11473 raft_consensus.cc:2368] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11.
I20260709 13:16:41.260089 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.252497 15313 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:41.199497 15272 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.296555 11468 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 3: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.255236 15318 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:41.323046 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63021.1us
W20260709 13:16:41.329012 11473 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1544) took 2213 ms (client timeout 2051 ms). Trace:
I20260709 13:16:41.327178 12083 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:41.329568 11473 rpcz_store.cc:269] 0709 13:16:39.108727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.299548 (+190821us) service_pool.cc:224] Handling call
0709 13:16:41.280011 (+1980463us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:41.322230 (+ 42219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11228928}
W20260709 13:16:41.332170 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 78446.5us
I20260709 13:16:41.192381 15309 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:41.334197 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:41.336745 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:41.339377 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 214246us
I20260709 13:16:41.340966 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.325449 15016 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:41.327502 15388 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:41.329020 15318 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.343214 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:41.339823 15354 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.346122 12123 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:41.350140 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.352272 12116 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:41.284300 15177 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.353005 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.353384 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.380332 11471 raft_consensus.cc:2376] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7. Re-sending same reply.
W20260709 13:16:41.380813 11471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1545) took 2271 ms (client timeout 2076 ms). Trace:
I20260709 13:16:41.338714 15373 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:41.333662 15418 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.374066 11397 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:41.375759 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.377359 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.384162 11546 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
W20260709 13:16:41.381134 11471 rpcz_store.cc:269] 0709 13:16:39.109014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.302620 (+193606us) service_pool.cc:224] Handling call
0709 13:16:41.353186 (+2050566us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:41.380746 (+ 27560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4401920}
I20260709 13:16:41.384289 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:41.351696 15388 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:41.378377 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:41.382687 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.387295 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.387434 11549 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:41.389287 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.388471 11468 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:41.390722 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46686.3us
I20260709 13:16:41.390919 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:41.385087 11473 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.392347 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.392959 11397 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:41.393390 12115 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:41.344864 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 94892.7us
I20260709 13:16:41.262934 15395 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.390084 15388 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.413641 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.415899 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.386344 15342 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.393204 15373 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:41.463590 15373 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:41.421542 15448 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:41.468163 11398 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:16:41.422562 15453 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:41.427703 11472 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:41.427611 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.434445 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.447144 15402 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:16:41.455528 15458 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.447388 15324 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.457533 15418 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.459568 15388 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.460532 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.460846 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.470403 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.462716 15423 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:41.519476 11476 raft_consensus.cc:2376] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7. Re-sending same reply.
W20260709 13:16:41.483166 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 48888us
I20260709 13:16:41.520206 15453 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.521000 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.522183 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.483989 15458 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.485836 11393 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:41.525750 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.526363 11393 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.532241 15146 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.533084 15146 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:41.546947 15248 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:41.547490 15318 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.549017 15453 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:41.553051 12114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 605) took 1748 ms. Trace:
I20260709 13:16:41.555935 15324 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:41.500567 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.557512 11471 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:41.509840 12118 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:41.511071 11468 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:41.559194 15417 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.561192 11400 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:16:41.561379 15417 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:41.562299 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:41.564265 11542 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:41.563972 15418 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:41.553524 12114 rpcz_store.cc:276] 0709 13:16:39.804965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.815046 (+ 10081us) service_pool.cc:224] Handling call
0709 13:16:39.815277 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.155296 (+340019us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:40.281716 (+126420us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.391076 (+109360us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.397125 (+  6049us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.397177 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:40.397219 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.072611 (+675392us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b1800467620. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:41.552735 (+480124us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54006c0f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:41.552802 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.552967 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4577,"prepare.queue_time_us":164374,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":10614,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":6232,"spinlock_wait_cycles":789977472,"thread_start_us":225130,"threads_started":2,"wal-append.queue_time_us":250618}
W20260709 13:16:41.566151 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47723.5us
I20260709 13:16:41.570577 15418 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:41.571630 15392 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:41.572249 15375 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.572257 14648 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:41.572213 15392 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:41.576469 15392 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.577152 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.577848 15403 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:41.578091 15392 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.542676 15344 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.578272 15403 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:41.580586 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.581037 15403 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.581697 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.582949 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:41.583315 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.583737 12116 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.515369 15309 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.584302 15355 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:41.585247 14893 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:41.585907 15309 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.586690 11473 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:41.587986 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:41.583855 11543 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.588696 11473 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:16:41.486123 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 88509.6us
I20260709 13:16:41.588153 11547 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:41.578415 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:41.592751 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.594177 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 550) took 1348 ms. Trace:
W20260709 13:16:41.592043 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 130937us
I20260709 13:16:41.446707 11397 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:41.597514 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48200 (request call id 19) took 3543 ms. Trace:
I20260709 13:16:41.598887 15320 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.599220 11395 raft_consensus.cc:2410] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:41.600152 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.600867 11395 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:41.604305 15320 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:41.605284 15320 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:41.605865 15320 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:41.606945 15320 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.608704 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.610224 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:41.612356 11545 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:41.619531 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.620293 11545 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:41.613703 11397 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.594602 12122 rpcz_store.cc:276] 0709 13:16:40.245688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.245902 (+   214us) service_pool.cc:224] Handling call
0709 13:16:40.246193 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:40.289935 (+ 43742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:40.289985 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:40.300811 (+ 10826us) log.cc:844] Serialized 26 byte log entry
0709 13:16:40.500775 (+199964us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:40.500838 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.800621 (+299783us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b54005e1ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:40.800680 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:41.593294 (+792614us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b24003e3738. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686afb0 00007f8da94e1488 00007f8da94e1be8 00007f8da94b9398 00007f8da94b476a 00007f8da94b3bb6
0709 13:16:41.593945 (+   651us) raft_consensus.cc:1685] finished
0709 13:16:41.593967 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.594105 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115900544,"thread_start_us":178272,"threads_started":1,"wal-append.queue_time_us":223696}
I20260709 13:16:41.595773 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41380 (request call id 513) took 1091 ms. Trace:
I20260709 13:16:41.622359 11551 rpcz_store.cc:276] 0709 13:16:40.504821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.504936 (+   115us) service_pool.cc:224] Handling call
0709 13:16:41.595578 (+1090642us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400134054. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:41.595714 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529482496}
I20260709 13:16:41.625257 11551 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.628057 11551 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:16:41.626556 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54593us
I20260709 13:16:41.629470 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.616109 11549 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:16:41.617090 11393 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:41.613113 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:41.588652 12123 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:41.598495 11396 rpcz_store.cc:276] 0709 13:16:38.054344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.069012 (+ 14668us) service_pool.cc:224] Handling call
0709 13:16:41.597299 (+3528287us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003101d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9913853 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:41.597410 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518086528}
I20260709 13:16:41.631428 11549 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:41.635044 11395 raft_consensus.cc:1275] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:16:41.636260 11395 pending_rounds.cc:85] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 8
I20260709 13:16:41.637353 11395 pending_rounds.cc:107] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:16:41.637739 11395 raft_consensus.cc:2889] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:41.618186 11543 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:41.641256 12114 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:41.642530 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:41.643360 11545 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.644235 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:41.643241 14648 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:41.669202 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.668602 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.669108 14648 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:41.669739 12085 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.3:46781 timed out after 1.592s (SENT) [suppressed 37 similar messages]
I20260709 13:16:41.669914 14648 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:41.670701 14648 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:41.544340 15099 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.641358 14893 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:41.689997 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:41.693322 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.710831 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.727300 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 205146us
I20260709 13:16:41.728368 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:41.735852 11401 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.666047 15403 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.658049 15097 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:16:41.749501 12116 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:41.752398 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:41.739938 14648 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.753779 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.643097 15364 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.655164 15342 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:16:41.688692 15450 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:41.671869 15417 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:16:41.773911 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.774606 11473 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.775120 15438 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.775838 15438 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.672219 15423 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:16:41.660300 15129 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.767529 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.643312 15436 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.781806 11469 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.738896 14893 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:41.792807 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 551) took 1546 ms. Trace:
I20260709 13:16:41.793224 12113 rpcz_store.cc:276] 0709 13:16:40.246645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.247595 (+   950us) service_pool.cc:224] Handling call
0709 13:16:40.262451 (+ 14856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:40.517134 (+254683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.792563 (+1275429us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54004c1b54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:41.792747 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567764864}
I20260709 13:16:41.793941 14648 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.790977 15402 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:41.796151 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.797534 15097 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:41.798153 15097 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:41.798588 15344 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:41.799248 15097 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:41.799629 15097 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.800364 15097 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.802433 15097 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:41.799633 15344 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:41.804956 15344 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.805922 15344 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.807739 15344 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:41.809250 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.810876 15417 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.811390 15417 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.811795 15417 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.812294 15364 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:41.813362 15393 consensus_meta.cc:287] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 13:16:41.757560 15146 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:41.816319 15146 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.816692 15146 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.818193 15436 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.764876 15450 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.819381 15450 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.702865 15296 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.821292 15312 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:41.822240 15312 consensus_queue.cc:1243] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:41.822438 11470 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:41.823344 12123 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:41.824119 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.824621 14893 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.826470 15129 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:41.828459 15423 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:41.828857 15423 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.829041 15248 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:41.829636 15423 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.837857 15402 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:41.838809 15402 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:41.831873 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:41.832245 11469 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:41.846450 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:16:41.851112 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 182617us
I20260709 13:16:41.851830 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:41.848659 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:41.875674 14648 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.876098 14648 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:41.876186 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 44375.8us
I20260709 13:16:41.876448 14648 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.877044 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:16:41.891728 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 138033us
I20260709 13:16:41.892715 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.894268 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.894944 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:41.878835 15177 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:41.916853 14648 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:41.880450 15423 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.880355 15342 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:41.926220 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.931270 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:41.941728 14648 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:41.950596 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.950703 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.951999 11393 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.952519 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.953671 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.954041 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.950496 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.959261 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.925719 15403 raft_consensus.cc:2764] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:16:41.916303 15452 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:41.971998 11548 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:41.961447 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.962699 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.973755 11476 raft_consensus.cc:2376] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:41.925401 15354 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:41.967366 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.967669 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.968112 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:41.929658 15382 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.929766 15177 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.976532 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.984123 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.984026 11545 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:41.985416 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.986514 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.955415 15418 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:41.955957 15436 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:41.992331 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.989423 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:41.995298 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:41.996268 11473 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.960821 15342 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:41.994860 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.003366 11476 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:41.993438 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.005081 11469 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:16:42.006174 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:41.976464 15382 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:41.932565 15392 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:16:42.016192 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.016882 11473 raft_consensus.cc:2376] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:41.990532 15099 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:41.997061 15418 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:42.017851 15459 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:42.015587 15382 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:42.044305 15465 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:16:42.046005 15463 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:16:42.068035 15342 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.022135 15438 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:42.081537 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 126573us
I20260709 13:16:42.081537 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2304) took 1220 ms. Trace:
I20260709 13:16:42.086725 11470 rpcz_store.cc:276] 0709 13:16:40.860858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.103540 (+242682us) service_pool.cc:224] Handling call
0709 13:16:41.129613 (+ 26073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.822911 (+693298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.081219 (+258308us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54003895d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:42.081465 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592913280}
I20260709 13:16:41.994174 15177 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:42.124496 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.133903 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:42.136847 11470 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:42.137454 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.091387 15417 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.158070 11470 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:42.112725 15416 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:16:42.163928 15399 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.117168 15418 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:42.182144 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 156023us
I20260709 13:16:42.197099 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:42.212584 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.176358 15450 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.215785 11397 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:42.217872 15417 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:42.218544 15402 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:42.219403 15462 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.449s	user 0.001s	sys 0.003s
I20260709 13:16:42.219652 15402 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:42.220295 15481 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:42.220825 15481 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:16:42.221031 15469 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:16:42.221711 15481 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.178835 15399 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.224678 15399 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:42.227815 15458 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:42.229792 15382 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.230695 15482 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:16:42.231992 11470 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:42.232969 11400 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:42.234472 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.239894 15422 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:42.240259 15422 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:42.241074 15422 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.241735 11468 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:16:42.242344 15422 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:42.242316 15482 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.253165 11396 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:42.256919 15146 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:42.258630 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.260180 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.266311 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.263926 15482 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:16:42.271701 15483 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.285625 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.299432 11544 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.280004 15418 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.287501 15395 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:42.289328 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2291) took 1217 ms. Trace:
I20260709 13:16:42.303784 15146 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:42.293124 15327 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:42.304720 15327 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:42.304760 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.305927 15318 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:42.305682 15327 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.303381 11542 rpcz_store.cc:276] 0709 13:16:41.071520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.084807 (+ 13287us) service_pool.cc:224] Handling call
0709 13:16:41.090527 (+  5720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.564648 (+474121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.289078 (+724430us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54002076d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:42.289271 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088605440}
I20260709 13:16:42.309017 15327 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:42.314838 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.316529 11544 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:42.317931 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:42.320768 15418 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:42.324254 15382 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:42.324584 15348 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:42.325050 15348 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:42.325886 15348 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.326521 15450 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:42.333190 15418 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:42.337263 11474 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:16:42.337913 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2314) took 1209 ms. Trace:
I20260709 13:16:42.338172 11474 rpcz_store.cc:276] 0709 13:16:41.128723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.520326 (+391603us) service_pool.cc:224] Handling call
0709 13:16:42.336735 (+816409us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4e9c 00007f8da67f4a5b 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:42.337021 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.337657 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.337867 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77070336}
I20260709 13:16:42.339887 15327 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:42.332690 15348 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.348080 15385 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:42.348562 15385 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:42.353893 11469 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:42.354766 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.356190 11472 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:16:42.356565 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.350534 15385 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.359736 15385 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:42.360942 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:42.362088 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:42.384218 12122 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.362156 12113 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:42.362326 11468 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:42.385588 11546 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:16:42.362895 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:42.363188 11550 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:42.363449 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:42.364476 11474 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:42.335309 15418 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.389511 11474 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:42.386686 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.357930 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:42.390729 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:42.368800 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.372124 15461 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:42.375690 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:42.393224 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.377103 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.394692 11545 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:42.379809 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 168883us
I20260709 13:16:42.358791 11477 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:42.360158 15379 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:42.368233 11548 raft_consensus.cc:1240] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:42.376235 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.396620 15391 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:42.396685 15420 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:42.397387 11472 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:42.401417 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.401868 11401 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:42.400480 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:42.402040 11546 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:16:42.428635 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.508996 15336 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:42.509923 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.510198 15312 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:42.510562 11472 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:42.510681 15312 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:42.403337 15461 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.509084 15420 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:16:42.511288 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 108014us
I20260709 13:16:42.526728 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:42.511763 15327 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:42.512025 11289 maintenance_manager.cc:419] P d085878dd61a4a3984dff72a53c92b4d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033369
I20260709 13:16:42.502220 15393 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.532079 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.496996 15422 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:42.539776 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.540470 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.542059 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:42.542415 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:42.543028 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:42.546389 11472 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:42.393003 15318 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:16:42.548537 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.503132 15284 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.551587 11399 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:42.549105 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:42.558950 11393 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:42.561942 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:42.565099 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.567765 11399 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:16:42.560971 11516 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:42.573297 11399 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:42.534971 15399 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:42.590008 11472 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:42.544389 15385 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.534900 15366 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:16:42.553918 15438 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.619632 12116 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:42.545434 15422 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:42.406350 15379 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.565935 15452 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.617012 12122 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:16:42.534502 15344 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:16:42.534754 15423 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:16:42.543257 15146 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:16:42.533761 15311 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.080s
W20260709 13:16:42.630259 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.598277 15399 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.550215 15393 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:42.617049 15448 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:42.569857 15466 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:16:42.713006 11546 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
W20260709 13:16:42.704201 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 155148us
I20260709 13:16:42.623602 15327 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.623775 15312 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.618526 15385 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.630077 14893 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:42.616431 15382 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:16:42.702390 15399 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:42.615320 15129 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:42.721442 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.720227 15458 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.692669 15450 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.733958 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.737046 11476 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:42.742323 15311 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.723759 15146 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:16:42.735950 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.745827 15320 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.745923 15474 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.747754 15421 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.748241 11471 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:42.748848 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1555) took 1617 ms. Trace:
I20260709 13:16:42.748272 15421 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.748479 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.751961 15474 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.752681 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:42.753271 12118 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.753182 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.755431 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:42.755843 15474 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:42.757532 15421 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:42.758641 15284 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:42.761111 15284 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:42.761519 15284 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:42.763312 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:42.749178 11471 rpcz_store.cc:276] 0709 13:16:41.132951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.558266 (+425315us) service_pool.cc:224] Handling call
0709 13:16:42.747777 (+1189511us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:42.747950 (+   173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.748629 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.748798 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131561856}
I20260709 13:16:42.764621 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:42.766885 11471 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:42.769723 15423 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:42.771272 15146 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:42.771297 15437 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:42.768934 15474 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:42.771858 15489 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.389s	user 0.001s	sys 0.000s
I20260709 13:16:42.784627 12083 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:42.765244 11470 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:42.790864 11471 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:42.795217 12118 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:42.800678 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:42.804656 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.736028 14893 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:42.773027 15493 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:42.813367 15366 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:42.725075 15327 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.783164 15359 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:16:42.820633 11402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1618) took 1844 ms (client timeout 1855 ms). Trace:
W20260709 13:16:42.852808 11402 rpcz_store.cc:269] 0709 13:16:40.976085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.976209 (+   124us) service_pool.cc:224] Handling call
0709 13:16:42.820355 (+1844146us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540026ff54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:42.820535 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241336704}
I20260709 13:16:42.822496 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 1953) took 1909 ms. Trace:
I20260709 13:16:42.827965 15483 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.827976 15466 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.762434 15284 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.835275 15080 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:42.869978 15080 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:42.736769 15453 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:42.845427 11546 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:42.870846 15080 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.872030 15080 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:42.872175 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:42.872848 11546 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:16:42.873335 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.815068 15296 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:42.875391 15296 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:42.875867 15296 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:42.876368 15327 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:42.876709 15296 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:42.849275 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 288407us
I20260709 13:16:42.851480 11547 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:42.854233 11398 rpcz_store.cc:276] 0709 13:16:40.912729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.915251 (+  2522us) service_pool.cc:224] Handling call
0709 13:16:40.915702 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:41.468618 (+552916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.822186 (+1353568us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b5400349254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:42.822429 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269935872}
I20260709 13:16:42.856920 11549 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:42.857801 15426 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:42.858794 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.881943 11544 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:42.859859 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:42.883620 11395 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
W20260709 13:16:42.860116 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 59561.3us
I20260709 13:16:42.864782 11470 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:42.867287 11548 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:42.868057 15379 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:42.869045 15344 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.868732 14893 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.732039 15422 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.783812 15458 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.783648 15450 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.873660 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:42.772500 15437 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:42.890388 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:42.891660 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:42.893604 11549 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:42.921518 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:42.880843 15129 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.922261 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:42.858894 15423 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:16:42.923945 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 139389us
I20260709 13:16:42.810545 15146 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.102s
W20260709 13:16:42.946163 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.864044 15483 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.924487 15461 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:42.947736 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1631) took 1610 ms. Trace:
I20260709 13:16:42.925765 15284 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.881642 15466 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.932068 15422 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:42.918749 15437 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:42.932924 15097 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.909937 15344 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.930208 15355 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:42.933727 15311 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:42.928134 15329 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 13:16:43.016598 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.933341 15500 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:42.935362 15248 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:16:42.907963 15426 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:42.949195 15417 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:43.008528 15355 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.049935 15355 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.050410 15426 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.038005 15393 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.044803 15500 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:43.043962 15421 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:16:43.016896 15097 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.052088 15500 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:42.971444 15362 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.104s
W20260709 13:16:43.068508 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.069854 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:43.006119 11394 rpcz_store.cc:276] 0709 13:16:41.337120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.367761 (+ 30641us) service_pool.cc:224] Handling call
0709 13:16:41.368225 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.357421 (+989196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.357450 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.362831 (+  5381us) log.cc:844] Serialized 52 byte log entry
0709 13:16:42.383839 (+ 21008us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:42.383881 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.833253 (+449372us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b5400348354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:42.833319 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:42.941718 (+108399us) raft_consensus.cc:1685] finished
0709 13:16:42.941739 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:42.947650 (+  5911us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4227,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":2274095488,"thread_start_us":20587,"threads_started":1,"wal-append.queue_time_us":238094}
I20260709 13:16:43.073571 11393 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:43.073915 11399 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:43.063387 15392 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.011s	sys 0.015s
W20260709 13:16:43.074646 15392 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.011s	sys 0.015s
I20260709 13:16:42.969889 15379 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:16:43.077344 15379 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.077965 15379 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.081094 15379 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.071117 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:43.093737 11402 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:43.102918 11473 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.014s	sys 0.002s
W20260709 13:16:43.103498 11473 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.014s	sys 0.002s
I20260709 13:16:43.098377 15453 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:43.097626 15402 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:16:43.104609 15453 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:43.104955 15402 consensus_queue.cc:1243] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:16:43.092459 15494 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:16:43.057606 15385 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.092322 15097 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:43.115808 15436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.094s	user 0.008s	sys 0.014s
I20260709 13:16:43.129911 12122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:56142
I20260709 13:16:43.105214 15483 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:43.134219 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 203150us
I20260709 13:16:43.133720 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:43.132123 12122 raft_consensus.cc:606] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:43.137009 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:43.137645 11545 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:43.135821 12122 raft_consensus.cc:614] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:43.188287 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1580) took 1223 ms. Trace:
W20260709 13:16:43.131796 15436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.008s	sys 0.015s
I20260709 13:16:43.198614 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 1962) took 1249 ms. Trace:
I20260709 13:16:43.109156 15474 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:16:43.217931 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.107779 15311 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 13:16:43.312119 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 308107us
I20260709 13:16:43.134390 15499 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.209898 11396 rpcz_store.cc:276] 0709 13:16:41.949307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.949448 (+   141us) service_pool.cc:224] Handling call
0709 13:16:43.198390 (+1248942us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540014aad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:43.198539 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663186816}
I20260709 13:16:43.328019 12085 leader_election.cc:400] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11.
I20260709 13:16:43.196413 11473 rpcz_store.cc:276] 0709 13:16:41.964708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.017571 (+ 52863us) service_pool.cc:224] Handling call
0709 13:16:42.028208 (+ 10637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.173075 (+144867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.173106 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.173531 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:16:42.276941 (+103410us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:42.836795 (+559854us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b180041d220. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:43.124814 (+288019us) log.cc:844] Serialized 14 byte log entry
0709 13:16:43.125850 (+  1036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.125919 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.125963 (+    44us) raft_consensus.cc:1685] finished
0709 13:16:43.188051 (+ 62088us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.188228 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6783,"prepare.queue_time_us":43548,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":82734,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":25575,"spinlock_wait_cycles":2330880,"thread_start_us":997902,"threads_started":3,"wal-append.queue_time_us":122995}
I20260709 13:16:43.328619 12085 leader_election.cc:403] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:16:43.317376 15489 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:16:43.098649 15437 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:43.338609 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 333978us
I20260709 13:16:43.303275 15436 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:43.187834 15097 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 13:16:43.316933 15505 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.200809 15426 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 13:16:43.327790 15379 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.324478 15314 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.339990 15450 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.354032 15146 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.358352 15311 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.358536 15362 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.136229 15453 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.383085 15437 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.398442 15518 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.333456 15499 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.419504 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:43.427039 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:43.431105 15497 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.364s	user 0.000s	sys 0.001s
I20260709 13:16:43.415030 15311 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.431979 15416 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:43.412619 15344 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.424352 15466 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:43.438022 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.410590 15146 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.431125 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:43.440151 15502 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:16:43.441069 15512 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:16:43.441135 15503 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11.
I20260709 13:16:43.444468 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:43.432577 15436 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.446012 11549 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:43.469887 11549 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:43.395900 15379 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.455542 11548 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:43.382283 15505 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:43.472102 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 144244us
W20260709 13:16:43.459159 15080 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.001s
I20260709 13:16:43.470732 11549 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.415742 15362 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.409439 15314 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.451581 15417 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:16:43.424149 15518 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.480858 15416 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:43.545130 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.480751 15177 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.480445 15344 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.553228 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2333) took 1165 ms. Trace:
I20260709 13:16:43.470825 15355 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.555051 11474 rpcz_store.cc:276] 0709 13:16:42.388165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.551255 (+163090us) service_pool.cc:224] Handling call
0709 13:16:42.551595 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.565624 (+ 14029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.565657 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.566182 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:16:43.006864 (+440682us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:43.006902 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.006971 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.552914 (+545943us) raft_consensus.cc:1685] finished
0709 13:16:43.552944 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.553146 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13665,"raft.queue_time_us":13775,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":12546,"thread_start_us":434168,"threads_started":1,"wal-append.queue_time_us":552802}
I20260709 13:16:43.575208 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1633) took 1930 ms. Trace:
I20260709 13:16:43.575824 11397 rpcz_store.cc:276] 0709 13:16:41.645359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.646196 (+   837us) service_pool.cc:224] Handling call
0709 13:16:41.646961 (+   765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.216319 (+569358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.572243 (+1355924us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54001261d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:43.575119 (+  2876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307585920}
I20260709 13:16:43.546380 15416 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:43.591176 15296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.032s	sys 0.018s
I20260709 13:16:43.589805 15416 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:43.560072 15177 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.590564 15509 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.572795 15344 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:43.632479 15296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.032s	sys 0.018s
I20260709 13:16:43.603451 15514 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.597268 15499 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:43.642119 15513 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:16:43.640851 15525 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:43.645759 15284 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.003s
I20260709 13:16:43.604291 15421 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.674114 15421 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.655262 14893 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.676465 15421 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.680732 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:43.681515 11548 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:43.683308 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 209381us
W20260709 13:16:43.666501 15399 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.004s
I20260709 13:16:43.676309 14893 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:43.671733 15504 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:16:43.679689 15521 log.cc:927] Time spent T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:16:43.685720 15466 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:43.697202 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:43.662528 15514 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.667636 15296 consensus_queue.cc:1048] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:16:43.686070 15146 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:43.684258 15499 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:43.644891 15509 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.679037 15525 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:43.711488 15373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.245s	user 0.000s	sys 0.061s
I20260709 13:16:43.688408 15399 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.739094 15362 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:43.755581 15373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.289s	user 0.000s	sys 0.062s
I20260709 13:16:43.778441 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:43.779314 11542 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:43.700780 15466 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.784641 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2338) took 1065 ms. Trace:
I20260709 13:16:43.785210 11475 rpcz_store.cc:276] 0709 13:16:42.719291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.768020 (+ 48729us) service_pool.cc:224] Handling call
0709 13:16:42.768290 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.768611 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.768639 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.769121 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:16:43.101749 (+332628us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:43.101790 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.101854 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.784347 (+682493us) raft_consensus.cc:1685] finished
0709 13:16:43.784378 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.784564 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21130,"raft.queue_time_us":17669,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9789,"thread_start_us":332285,"threads_started":1,"wal-append.queue_time_us":592098}
I20260709 13:16:43.798588 11401 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:43.782948 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 423514us
I20260709 13:16:43.701597 15526 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:16:43.759022 15355 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:43.757376 15314 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.756897 15500 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:43.819620 15531 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:16:43.819958 15450 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.820539 15450 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.821974 15525 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:43.822728 15450 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.824195 15509 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.783048 15530 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.824975 15530 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:43.826941 15534 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:16:43.797467 15311 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.829413 15379 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.830086 15379 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:43.830446 15379 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:43.831619 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2335) took 1302 ms. Trace:
I20260709 13:16:43.831310 15379 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.832613 15379 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.832005 11472 rpcz_store.cc:276] 0709 13:16:42.529293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.591788 (+ 62495us) service_pool.cc:224] Handling call
0709 13:16:42.614334 (+ 22546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.614750 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.614777 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:43.440189 (+825412us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:43.442692 (+  2503us) log.cc:844] Serialized 26 byte log entry
0709 13:16:43.635591 (+192899us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:43.635637 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.635707 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.831287 (+195580us) raft_consensus.cc:1685] finished
0709 13:16:43.831319 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.831555 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3552,"raft.queue_time_us":82,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":15032,"spinlock_wait_cycles":5595392,"thread_start_us":192540,"threads_started":1,"wal-append.queue_time_us":247115}
W20260709 13:16:43.836127 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 130541us
I20260709 13:16:43.838034 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:43.844226 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:43.746328 15499 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.808333 15362 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.809516 15528 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.844954 11542 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:43.847913 12116 raft_consensus.cc:1240] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:16:43.812901 15490 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:43.847723 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:43.849100 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:43.851329 11475 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
I20260709 13:16:43.851539 11473 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:43.853461 12116 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:43.813335 15518 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.852293 15362 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.859203 15296 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:43.860903 15416 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.861402 15458 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.836817 15532 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.865803 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:43.845757 15509 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:43.880203 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:43.879283 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:43.850952 15490 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:43.886212 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:43.887066 15416 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.896085 12122 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:43.853590 15344 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.837505 15500 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.900797 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:43.904399 12116 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
W20260709 13:16:43.912062 15529 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:43.912313 15505 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:43.920007 15422 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.000s
I20260709 13:16:43.920604 15422 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.921105 15422 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.923138 15536 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:16:43.924249 15528 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:43.875373 15466 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.937799 15436 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.941574 15544 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:43.942457 15544 consensus_queue.cc:1243] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:16:43.936798 15422 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.882341 15438 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:43.943286 15438 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:43.944155 15438 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.945142 15438 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.945369 15500 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:43.946182 15490 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.947432 15499 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.896261 15532 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.935942 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:43.948390 15399 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.948843 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:43.948992 15399 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:43.949371 15532 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:43.949465 15399 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.949957 15518 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:43.950616 15518 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.950007 15466 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:43.899410 15450 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:43.954706 12083 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:43.954396 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:43.961256 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:43.959981 15450 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.970016 12116 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:43.971427 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:43.971916 15450 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.972328 11472 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:43.947432 15490 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:43.976693 15526 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:43.977141 15526 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.977526 15526 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:43.950970 15518 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:43.976701 15314 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:43.979246 15314 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.979588 15314 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:43.980379 15437 consensus_meta.cc:287] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 13:16:43.984661 15362 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.985592 11544 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:43.988927 15177 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.989568 15538 consensus_queue.cc:1059] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:16:43.899642 15509 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:43.992583 15509 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:43.992964 15312 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:43.960273 15466 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.958166 15436 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:43.995711 15466 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:43.949919 15399 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.996120 15436 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:44.000772 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 146055us
I20260709 13:16:44.000217 15248 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:44.013842 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.013108 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.014681 11545 raft_consensus.cc:2410] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:16:44.014894 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:44.013665 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:44.015635 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.016609 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.017561 15536 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:43.957782 15418 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.018442 11547 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:43.956610 15422 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.018973 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.022224 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:44.025921 11542 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:44.026542 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:44.030306 15517 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.589s	user 0.001s	sys 0.002s
I20260709 13:16:44.036514 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.037209 11546 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:44.038275 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:44.019766 15536 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.014230 15248 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:44.043280 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.049373 11542 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:44.036279 15379 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:44.054309 15536 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.044548 15503 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:44.018505 15526 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.075490 15248 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.053690 15344 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:44.038301 15418 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.036695 15422 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.095001 15129 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:44.099656 14893 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.104430 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.111290 12116 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:44.113015 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:44.114282 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.114666 15538 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:44.115159 15538 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.115229 15524 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.115635 15538 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:44.115816 15524 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.118140 15524 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:44.086817 15546 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:44.125706 15418 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.129630 15530 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.089669 15344 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.131714 15344 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.132316 15553 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.132823 15553 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.110802 15129 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:44.135331 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.156719 11473 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:44.135942 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 2019) took 1936 ms. Trace:
I20260709 13:16:44.157723 11550 rpcz_store.cc:276] 0709 13:16:42.199673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.199829 (+   156us) service_pool.cc:224] Handling call
0709 13:16:42.200136 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.387987 (+187851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.135726 (+1747739us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540033d6d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:44.135886 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374551936}
I20260709 13:16:44.158269 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.143069 11542 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:44.143394 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.143718 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.144529 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.151491 11397 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:44.152982 11402 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:44.153564 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.153764 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.154032 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.166185 11475 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:44.166611 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.146190 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.167375 11542 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.167701 11548 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 3.
I20260709 13:16:44.154754 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.168274 11394 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:44.172312 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.174239 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:44.176945 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:44.177970 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.185073 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.186022 12118 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.187937 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.186282 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.154130 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.188572 11396 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:44.188680 11473 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:44.188719 11393 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:44.190412 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.191224 11397 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:16:44.194530 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.196095 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.198736 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.199772 11542 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 3.
W20260709 13:16:44.200559 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.200991 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.203027 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.203439 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.205018 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.214918 11399 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:44.213442 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:44.214313 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.216542 11393 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:44.211220 11542 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:44.217901 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.212127 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.219933 11398 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.217015 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.220839 11396 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:44.212769 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.222683 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.136157 15354 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:44.223372 11548 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:44.147704 15386 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:44.231549 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.231805 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.232560 11547 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:44.227178 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.193367 15436 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.232715 11476 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:44.226114 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.134946 15553 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.186398 15421 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.203303 15450 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.196152 15342 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.202936 15362 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.196287 15382 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.195003 15426 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:44.241039 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.205303 15417 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.241905 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.256644 11547 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:44.243976 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.245641 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.195168 15509 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.225082 15418 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:44.227650 15354 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:44.237533 15329 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.258067 11548 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:44.205832 15518 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.230371 15386 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:44.285867 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.263257 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.286888 11546 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:44.224864 15318 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:44.140277 15129 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.245635 15296 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.289911 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.291585 11396 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:44.304057 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.304826 11468 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:44.307971 11468 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:44.309542 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.311228 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.283138 15553 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.312192 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:44.316134 11468 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.312263 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.317054 11546 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:44.312389 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.318411 11475 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:44.312599 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.242720 15436 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.328810 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.329706 11473 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:44.331280 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.332042 11475 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:44.332994 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.289328 15329 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:44.335537 15561 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:16:44.336647 15552 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.337184 15552 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.337244 15386 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.338745 15386 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.340166 15553 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.261305 15509 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.344683 15509 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.345634 15344 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.284986 15418 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.267376 15417 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.348863 15418 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.349212 15545 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.349769 15545 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.350358 15399 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.304618 15422 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.252795 15421 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.353266 15562 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.353960 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.354596 11393 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:44.355396 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.355155 15354 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.355984 11393 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:16:44.356559 15354 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.356845 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.357012 15536 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.357782 11393 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:44.254169 15450 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.359404 11393 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:44.360658 15450 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.259325 15342 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.263689 15426 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.261770 15362 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.261349 15382 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.363842 15571 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:44.364333 15571 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.364694 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.365265 15382 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.365486 11472 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:44.367161 11542 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:44.368835 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.370033 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.370707 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.371199 11398 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:44.370736 11397 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:44.363356 15426 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.346570 15538 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.333702 11475 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:44.376497 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.364715 15471 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:44.364703 15571 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:44.406749 11550 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10.
I20260709 13:16:44.342904 15553 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:44.409242 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:44.410357 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.410748 15537 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.411330 15537 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:44.416253 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.418030 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.421059 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.388139 11399 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.424614 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.390431 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.426409 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.427911 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.391193 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.393993 11472 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:44.397639 15344 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.398828 15399 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:44.399441 11401 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1635) took 1860 ms (client timeout 1838 ms). Trace:
I20260709 13:16:44.403148 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.336658 15329 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.387949 15355 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.433566 15471 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.443552 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.443387 15355 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.364351 15526 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.403961 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:44.444660 11546 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:44.441216 11401 rpcz_store.cc:269] 0709 13:16:42.538963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.541820 (+  2857us) service_pool.cc:224] Handling call
0709 13:16:44.399245 (+1857425us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003a7ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:44.399373 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2850147072}
W20260709 13:16:44.404243 15560 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:16:44.448331 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.404758 15552 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.380635 11476 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:44.411295 15538 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.433943 15379 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:44.429612 12113 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:44.430240 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.430477 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.433732 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.456343 11402 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:44.417311 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.434950 11397 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:44.435580 15373 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.433563 15416 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.437147 15436 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.437867 12115 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:44.437808 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.435278 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:44.442476 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.461068 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:44.442708 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.458395 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.447772 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.463428 11401 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.453015 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.427353 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.435099 15553 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.439919 15536 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:44.461192 12123 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:44.461982 11393 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.446702 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.464118 11399 raft_consensus.cc:2393] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.472353 11468 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:44.460935 12115 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:44.472227 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.477366 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:44.477947 11399 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.474207 12115 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.484172 12115 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:44.455183 15379 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.485351 11546 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 4.
I20260709 13:16:44.486585 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.486828 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.456959 15342 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.491652 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.493441 15399 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:44.494369 15426 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:44.499344 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.501507 15450 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:44.502101 15450 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:44.473074 15329 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:44.502918 15329 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:44.471560 15416 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.503124 15450 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.504246 15392 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:44.503964 15329 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.504802 15392 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:44.505147 15416 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.470971 15382 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:44.506114 15382 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.506471 15382 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:44.512830 15515 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.973s	user 0.000s	sys 0.004s
I20260709 13:16:44.460486 15373 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.514161 15373 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.514350 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2306) took 1671 ms. Trace:
I20260709 13:16:44.515666 15296 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.514930 11543 rpcz_store.cc:276] 0709 13:16:42.843093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.843665 (+   572us) service_pool.cc:224] Handling call
0709 13:16:42.860155 (+ 16490us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.860499 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.860533 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.884172 (+ 23639us) log.cc:844] Serialized 52 byte log entry
0709 13:16:43.316964 (+432792us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:43.317004 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.317071 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:44.512245 (+1195174us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:44.513912 (+  1667us) raft_consensus.cc:1685] finished
0709 13:16:44.513945 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.514268 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6348,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":5020544,"thread_start_us":392156,"threads_started":1,"wal-append.queue_time_us":558762}
I20260709 13:16:44.517935 11396 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:16:44.518313 11396 pending_rounds.cc:85] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 7
I20260709 13:16:44.518532 11396 pending_rounds.cc:107] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:16:44.519024 11396 raft_consensus.cc:2889] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:44.466015 15490 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.466442 15552 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:44.518465 11543 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:44.603384 15552 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:44.604388 15552 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:44.567929 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 74702.3us
I20260709 13:16:44.607582 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:44.608423 15309 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:44.609349 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.609833 15309 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.610280 15309 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.610630 15309 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.611194 15422 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:44.577031 15355 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:44.614082 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:44.614905 11399 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
W20260709 13:16:44.577244 15568 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:16:44.579020 15314 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.002s
I20260709 13:16:44.572285 15426 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.584321 15490 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.572710 15538 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:44.586385 15392 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.588930 15471 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.590683 15421 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.591369 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.592494 15466 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.593207 15403 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:44.594174 15511 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.033s	user 0.003s	sys 0.001s
I20260709 13:16:44.596758 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:44.597370 15483 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:44.646209 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2319) took 1784 ms. Trace:
I20260709 13:16:44.646646 11544 rpcz_store.cc:276] 0709 13:16:42.861950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.882653 (+ 20703us) service_pool.cc:224] Handling call
0709 13:16:42.882890 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.986002 (+1103112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.645945 (+659943us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540041c8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:44.646145 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2535135488}
I20260709 13:16:44.648661 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:44.620374 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 236032us
I20260709 13:16:44.649972 11473 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:44.618480 15417 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.622444 15423 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.616532 15336 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:44.483290 15553 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.654058 15318 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.657184 15336 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.657618 15318 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.659302 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:44.649281 15403 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.659847 15553 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.648850 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2343) took 1868 ms. Trace:
I20260709 13:16:44.617681 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:44.662074 15566 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:44.661285 15309 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.662478 15566 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.625643 15538 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.662863 15566 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.622638 15426 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.652379 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.602780 15373 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.652518 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.651773 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.666278 15426 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.666407 11401 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:44.666740 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.619174 15314 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.668974 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:44.669679 15146 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.669806 11516 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:44.670375 15146 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:44.670730 15146 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.671646 15146 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.672175 15490 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.672746 12119 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:44.656960 15423 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.674683 15423 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.552125 15379 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:44.667529 15355 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:44.668027 15373 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.671229 15438 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:44.660027 11398 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:44.661284 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50366us
I20260709 13:16:44.676098 15314 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.696501 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.664407 15309 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.702901 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.704656 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:44.706205 11544 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:44.706447 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.660539 11471 rpcz_store.cc:276] 0709 13:16:42.780341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.838947 (+ 58606us) service_pool.cc:224] Handling call
0709 13:16:42.839302 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.839760 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:42.839788 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.840386 (+   598us) log.cc:844] Serialized 52 byte log entry
0709 13:16:43.116418 (+276032us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:43.116476 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.123822 (+  7346us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:44.593535 (+1469713us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:44.648563 (+ 55028us) raft_consensus.cc:1685] finished
0709 13:16:44.648584 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.648795 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":887,"raft.queue_time_us":2211,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":13382,"spinlock_wait_cycles":46236544,"thread_start_us":275611,"threads_started":1,"wal-append.queue_time_us":641314}
I20260709 13:16:44.679389 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.708576 12122 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:44.681305 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.709834 12117 raft_consensus.cc:2410] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:44.659345 15336 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.710949 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.712180 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.713181 11471 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:44.716410 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.485086 15344 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.139s
I20260709 13:16:44.650429 15518 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.690101 11473 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:44.725195 11470 raft_consensus.cc:1217] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 9.5->[10.6-10.6]   Dedup: 10.6->[]
I20260709 13:16:44.725754 11470 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:16:44.690644 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 217346us
I20260709 13:16:44.727746 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.696753 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.637593 15466 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.699467 15490 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:44.701987 11399 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:44.728226 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:44.712875 11550 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:44.692955 15355 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.700836 15438 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.721335 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.723371 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.728140 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.730424 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:44.744496 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:44.748003 11543 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:44.713733 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.735411 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:44.751102 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:44.736815 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:44.752537 12116 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:16:44.720165 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.753285 11401 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:44.701171 15538 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.756191 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.732203 15309 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:44.760419 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:44.760871 11397 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.766605 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.761312 11395 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:44.762377 11396 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:44.772039 11476 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:44.731576 15437 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.774488 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.777421 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.778851 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.779161 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.780026 11393 raft_consensus.cc:2410] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:44.744390 15524 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.754837 15355 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:44.791357 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.792337 15382 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.793264 15359 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:44.758257 15518 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:44.793730 15530 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.796353 15466 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:44.750698 15474 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:44.750921 15528 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:44.755157 15438 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:44.799504 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.757989 15309 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.803938 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:44.758003 15573 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.806064 12119 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:44.806052 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.808923 11469 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:44.809005 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:44.723913 15461 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:16:44.721230 15571 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.839452 11469 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:44.799146 15436 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:44.799085 15422 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:44.845721 11468 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 4: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:44.781153 15437 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:44.845945 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.846724 11397 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:44.844455 15309 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.851292 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:44.842398 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:44.854193 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.854725 15483 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:44.856951 15571 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.842856 15528 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:44.812295 15403 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:44.842080 15336 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:44.856257 15575 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:44.813019 15509 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.858301 15528 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.842453 15573 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.786888 15514 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:44.836607 15568 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:16:44.876981 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66cb4598d33043fc8eb235647a242814"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:44.803851 15490 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:16:44.892313 11397 raft_consensus.cc:606] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.892776 11397 raft_consensus.cc:614] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:44.810726 15524 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.856529 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.843386 11469 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:44.838831 15097 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:44.807374 15518 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:44.895699 15518 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.855329 12112 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:44.889746 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:44.897688 11469 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.897944 11393 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:44.855377 15530 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:44.846207 15466 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:44.859961 15318 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:44.844703 15436 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:44.862046 15437 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:44.893429 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 142389us
I20260709 13:16:44.893671 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.901722 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.903772 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.904983 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:44.886370 15336 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:44.867530 15423 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:16:44.912779 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.898850 15097 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.896907 15518 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.900615 15509 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:44.917781 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:44.922309 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.923512 11469 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:44.924553 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2322) took 2569 ms. Trace:
I20260709 13:16:44.908721 15403 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.928020 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:44.925040 11469 rpcz_store.cc:276] 0709 13:16:42.354928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.384597 (+ 29669us) service_pool.cc:224] Handling call
0709 13:16:42.384860 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.808817 (+2423957us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000cfe20. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9890dc0 00007f8da991b1c0 00007f8da991c231 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c
0709 13:16:44.924094 (+115277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.924478 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587000192}
I20260709 13:16:44.940331 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:44.917052 15097 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.917083 15483 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.903182 15530 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:44.914050 15318 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 13:16:44.930358 15473 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:16:44.931592 15403 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.935606 15466 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:44.997059 15354 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:44.935987 15402 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:44.934580 15418 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.943714 15588 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:45.011974 15583 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:16:45.013526 15587 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:16:45.013814 15327 consensus_queue.cc:1175] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" responder_term: 11 status { last_received { term: 11 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 11 index: 9 } }
I20260709 13:16:45.003432 15593 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:45.003954 15403 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.006306 15466 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.009238 15402 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.008471 15354 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:45.015506 15471 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:45.028127 15524 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.024297 15466 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:45.031976 15574 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.531s	user 0.002s	sys 0.000s
I20260709 13:16:45.043337 15573 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.044430 15566 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.047156 15566 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.051749 15530 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.018100 15588 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:45.057080 15588 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:45.061535 15593 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:45.101132 11401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:45.103065 11401 raft_consensus.cc:606] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:45.103387 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 173680us
I20260709 13:16:45.103914 11401 raft_consensus.cc:614] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:45.105116 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.112396 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.112962 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.113718 11393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:45.113775 11401 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:45.114616 11393 raft_consensus.cc:606] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:45.115067 11393 raft_consensus.cc:614] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:45.115690 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.116566 11399 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.118683 11393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:45.120146 11393 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:45.127663 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:16:45.133873 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.139400 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:45.146175 15392 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.147158 15509 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:45.147886 15509 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.147403 15392 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.148434 15509 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.024217 15402 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.006201 15336 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.025465 15471 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:45.016693 15418 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:44.992853 15436 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.024775 15354 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.021939 15382 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:45.197219 11398 consensus_meta.cc:287] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.002s
I20260709 13:16:45.211309 15524 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.212174 15524 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.212744 12122 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:45.154938 15318 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.229983 11399 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
W20260709 13:16:45.220171 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 299372us
I20260709 13:16:45.214653 15490 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.238459 15600 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.239159 15600 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.241600 15600 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.243629 15318 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:45.244884 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 304503us
I20260709 13:16:45.246297 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:45.217612 15379 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.005s
W20260709 13:16:45.249030 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.251789 15596 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.221843 11398 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:45.227461 15418 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.214947 15524 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.215205 15146 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:45.263332 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.258786 15392 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:45.221699 15483 consensus_queue.cc:1175] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e9c2603c0bd4d25b713649b3890c46b" responder_term: 8 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:45.265280 15146 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.269471 15422 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:45.269304 15403 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.271168 15490 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.272125 15418 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.273907 15490 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.275442 15437 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.277962 11516 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.278275 15437 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.256947 15379 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.285827 15418 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.296428 15379 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.297641 12123 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:45.280076 15593 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:45.299885 15593 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.219843 15471 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.246806 15318 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.287304 15438 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.304932 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.305272 15438 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.306398 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:45.288358 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 149172us
I20260709 13:16:45.308787 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:45.309041 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:45.311295 15530 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:45.312021 15530 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:45.312913 15530 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.319504 15318 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:45.319643 15437 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:45.320261 15318 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:45.321466 15403 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.321880 15437 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.292918 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.321467 15318 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.322834 15437 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.325477 15437 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.278573 15309 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.322196 15403 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:45.301367 15593 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.340992 15354 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.305315 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.352293 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.353433 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.353467 11402 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
W20260709 13:16:45.357673 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.361263 15471 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.335238 15418 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.372773 15097 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.378726 11397 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:45.369304 15309 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.381317 15438 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.387023 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.394327 15402 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.397325 15318 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:45.397627 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.388267 15418 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.388078 15438 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.403774 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.408252 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.391222 15509 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.403599 15097 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.403760 15354 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.411180 11393 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:45.406355 15146 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:16:45.413936 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:45.414175 15146 consensus_queue.cc:1243] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:45.414688 12122 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.415406 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.415632 15466 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.416862 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.420667 15597 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:45.422422 15597 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:45.423771 15597 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.418584 15466 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.437330 15588 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.439296 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2324) took 2022 ms. Trace:
I20260709 13:16:45.444276 11548 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:45.453018 15402 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.455029 12122 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:45.439848 15588 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:45.442165 15603 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:16:45.454851 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:45.443398 15355 consensus_meta.cc:287] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.008s
I20260709 13:16:45.433346 15309 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:45.441319 15418 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.465972 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.471082 11394 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10.
I20260709 13:16:45.471869 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.476377 15545 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.473251 15355 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.494385 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:45.503748 15604 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:45.503957 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.504642 11548 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.505681 15438 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.464761 15309 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.499372 15099 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:45.501384 15355 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:45.496409 15579 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.590s	user 0.003s	sys 0.000s
I20260709 13:16:45.468192 15146 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:45.514402 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:45.516502 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.513487 15146 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.517797 15146 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:45.522204 11393 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:45.510571 15418 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:16:45.524775 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.527179 15588 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.527892 15588 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.530241 15588 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.527172 15099 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:45.535487 15099 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:45.536844 15099 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.537304 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:45.534520 15596 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:45.526743 15509 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:45.439774 11551 rpcz_store.cc:276] 0709 13:16:43.416695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.417470 (+   775us) service_pool.cc:224] Handling call
0709 13:16:43.417769 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.402312 (+984543us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b18003a0ae0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8da9911060 00007f8da9911bfb 00007f8da991c64b
0709 13:16:44.402430 (+   118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.402453 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.402881 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:16:44.627546 (+224665us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:44.627600 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.418184 (+790584us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54002076d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:45.420102 (+  1918us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.420145 (+    43us) raft_consensus.cc:1685] finished
0709 13:16:45.420152 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.439103 (+ 18951us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":221312,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":6848,"spinlock_wait_cycles":57202816,"thread_start_us":404491,"threads_started":2,"wal-append.queue_time_us":268193}
I20260709 13:16:45.507738 15471 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:45.549964 15606 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:16:45.555631 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:45.556495 15466 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:45.557090 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 85316.1us
I20260709 13:16:45.512804 15309 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:45.562949 11548 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:45.590143 11402 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:45.585947 15588 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:45.606082 11516 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:45.607394 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51810.9us
I20260709 13:16:45.592340 15466 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.610316 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.592095 15545 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.616583 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.614420 15471 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:45.623908 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.626296 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.627213 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.611611 15466 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.630053 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.613706 15588 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:45.594324 15530 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.633649 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.626605 11398 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:45.620721 15509 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:16:45.653259 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.654418 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.657842 11401 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:45.631915 15471 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:45.668097 12085 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10.
I20260709 13:16:45.668532 12085 leader_election.cc:403] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:16:45.668704 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:45.669118 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 76166.6us
I20260709 13:16:45.669486 11395 raft_consensus.cc:2368] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:45.668815 15336 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.670356 15336 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.671392 15526 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10.
I20260709 13:16:45.673147 15336 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.673900 11548 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:45.674926 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.675307 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 541) took 1305 ms. Trace:
I20260709 13:16:45.676057 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.676656 11548 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.679533 15382 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:45.680342 15354 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:45.680876 15354 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:45.681605 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.675631 11542 rpcz_store.cc:276] 0709 13:16:44.311027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.586410 (+275383us) service_pool.cc:224] Handling call
0709 13:16:44.586703 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.680158 (+ 93455us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:44.683300 (+  3142us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.793554 (+110254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:44.793777 (+   223us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:44.793915 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.445771 (+651856us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18001451e0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:45.596393 (+150622us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b54001ec8d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:45.603319 (+  6926us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.616077 (+ 12758us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24367,"prepare.queue_time_us":3524,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":76517,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":13734,"spinlock_wait_cycles":238008320,"thread_start_us":84952,"threads_started":1,"wal-append.queue_time_us":176694}
I20260709 13:16:45.683166 15524 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:45.683959 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.683816 15524 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.685101 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.685824 15524 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:45.686758 11549 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:45.691063 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.691115 15588 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.685844 11542 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:16:45.697254 15129 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.698349 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1607) took 1211 ms. Trace:
I20260709 13:16:45.698714 15471 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.698796 11477 rpcz_store.cc:276] 0709 13:16:44.486538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.487673 (+  1135us) service_pool.cc:224] Handling call
0709 13:16:44.488061 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.488463 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.488511 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:45.559648 (+1071137us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:45.560197 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.631513 (+ 71316us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:45.657825 (+ 26312us) log.cc:844] Serialized 14 byte log entry
0709 13:16:45.658140 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.664868 (+  6728us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.697798 (+ 32930us) raft_consensus.cc:1685] finished
0709 13:16:45.697899 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.698115 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18911,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":2608,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":56916,"spinlock_wait_cycles":295939968,"thread_start_us":70923,"threads_started":1,"wal-append.queue_time_us":134831}
I20260709 13:16:45.674714 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:45.697242 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.723743 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.689618 15146 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:45.727612 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.733832 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.665079 15509 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:45.682900 15354 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.745018 11542 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:45.747303 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:45.749601 15248 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:45.755089 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50653.5us
I20260709 13:16:45.759526 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:45.760298 11477 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:45.760712 15514 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:16:45.750368 11549 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:45.746182 11299 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.767431 15530 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:45.768514 15582 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.868s	user 0.001s	sys 0.000s
I20260709 13:16:45.768795 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:45.770027 12085 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10.
I20260709 13:16:45.772044 15379 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:45.776741 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:45.777500 11477 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:16:45.778469 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:45.661947 15545 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:45.780205 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 163778us
I20260709 13:16:45.770838 11299 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 2 to 5, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 5 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.808276 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2326) took 2330 ms. Trace:
I20260709 13:16:45.808758 11549 rpcz_store.cc:276] 0709 13:16:43.445401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.445560 (+   159us) service_pool.cc:224] Handling call
0709 13:16:43.445874 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.596254 (+2150380us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5800872cd0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da98750aa 00007f8da98f7447 00007f8da990ffe7 00007f8da99117de 00007f8da991bb49 00007f8da991c416 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4
0709 13:16:45.765995 (+169741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.775231 (+  9236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174002560}
I20260709 13:16:45.817924 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.785384 15379 raft_consensus.cc:697] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:45.828996 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.839071 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:45.831398 15452 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:16:45.845986 11548 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:45.838919 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:45.848127 11549 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.848019 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.848834 11550 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.895761 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:45.892553 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56108.3us
W20260709 13:16:45.897310 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 48680.4us
I20260709 13:16:45.897428 15536 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:45.837963 15099 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:45.908206 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.910429 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:45.927248 11516 leader_election.cc:400] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:45.927765 11516 leader_election.cc:403] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:45.883132 15593 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.928485 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.948437 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.953471 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.957114 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.958016 11551 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:45.930068 15146 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:45.946427 15354 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.964737 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:45.965827 15587 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:16:45.966090 15562 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.948995 15379 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:45.968568 11400 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:45.968992 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.971081 15537 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:45.971524 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 69430us
I20260709 13:16:45.971629 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:45.972893 15612 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:16:45.942407 15509 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:45.973352 11393 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:45.974072 15622 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:16:45.974882 15617 log.cc:927] Time spent T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:16:45.962805 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.948326 15348 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:45.976523 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:45.999572 11397 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:16:45.978648 11549 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
W20260709 13:16:46.000892 15624 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:16:46.001037 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.001624 11402 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:46.001924 11549 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:46.002820 11398 raft_consensus.cc:2393] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:46.002892 15621 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:46.004016 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.005488 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:45.986008 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:45.964767 15461 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.991372 11548 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.948658 15099 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:46.008258 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78828.4us
I20260709 13:16:46.011381 15545 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.015069 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.015931 11471 pending_rounds.cc:85] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 3
I20260709 13:16:46.016103 11477 raft_consensus.cc:2410] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:16:46.016285 11471 pending_rounds.cc:107] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:46.016662 11471 raft_consensus.cc:2889] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:46.017408 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:45.989943 15097 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.021502 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.022055 11393 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:46.023028 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.035703 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.011715 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.045320 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 764) took 1317 ms. Trace:
I20260709 13:16:46.046433 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:46.036576 11399 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:46.050676 11550 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:46.060137 11401 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:46.058634 12123 rpcz_store.cc:276] 0709 13:16:44.727409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.727521 (+   112us) service_pool.cc:224] Handling call
0709 13:16:46.045094 (+1317573us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b5400745254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:46.045243 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308556672}
I20260709 13:16:46.040928 11470 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.043607 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.046028 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 572) took 1594 ms. Trace:
I20260709 13:16:46.069428 11542 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:46.067646 12119 rpcz_store.cc:276] 0709 13:16:44.451159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.451304 (+   145us) service_pool.cc:224] Handling call
0709 13:16:46.045669 (+1594365us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54004de254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:46.045830 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485353344}
I20260709 13:16:46.057271 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.071697 11548 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:46.048409 15362 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.074214 15327 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:46.075603 15628 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:46.076151 15099 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.007493 15562 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.077384 11398 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:46.078351 12114 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:46.079731 15359 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:46.055868 15593 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:46.083702 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 172274us
I20260709 13:16:46.081396 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.083567 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.025924 15097 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.086234 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.083705 11547 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:46.087437 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:46.088836 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 90393us
I20260709 13:16:46.025820 15309 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.091722 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.070767 15348 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:46.106228 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.018507 15461 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.107838 11548 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:46.080147 15359 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.082001 15382 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.112071 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.114820 11474 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:46.112916 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.115427 11547 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:46.116577 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.067111 15466 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.117482 15359 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.122546 15382 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.128450 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.129655 11548 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:46.135942 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.133734 11470 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:46.139869 11399 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:46.141328 11476 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:46.142354 11475 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:46.139407 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.142935 11548 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:16:46.143059 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 96753.2us
I20260709 13:16:46.150430 11550 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:46.151038 11468 raft_consensus.cc:2368] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:16:46.134552 15327 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:46.151960 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.135527 15146 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.138105 15450 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.180301 15450 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.148072 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.146932 15473 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.158156 15146 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.163246 15436 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.169030 15552 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.186293 15552 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.188470 11547 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:46.188572 15552 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:46.189971 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43695.3us
I20260709 13:16:46.190375 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 532) took 1996 ms. Trace:
I20260709 13:16:46.191579 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.190788 11545 rpcz_store.cc:276] 0709 13:16:44.193918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.194092 (+   174us) service_pool.cc:224] Handling call
0709 13:16:44.194505 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.133884 (+939379us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:45.134593 (+   709us) log.cc:844] Serialized 14 byte log entry
0709 13:16:45.259498 (+124905us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.259566 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:45.259622 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.190007 (+930385us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54002067d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:46.190051 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.190210 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12701,"prepare.queue_time_us":50188,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":69417,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":17554,"spinlock_wait_cycles":1518952832,"thread_start_us":386480,"threads_started":2,"wal-append.queue_time_us":204516}
I20260709 13:16:46.192668 11542 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:46.193221 15473 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.195653 15473 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.197642 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.197885 15421 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:46.188678 15146 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.172796 15309 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.175406 15348 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.179055 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.184937 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.199079 11441 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.197830 15636 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:16:46.255127 15097 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.257269 11399 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.229424 11396 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:46.257944 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.157138 15327 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.238922 11394 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:16:46.258745 15327 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.262792 11474 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
W20260709 13:16:46.263316 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 115843us
I20260709 13:16:46.264901 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:46.192883 11367 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.266943 15097 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:46.220112 15466 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.267645 15474 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.268153 15474 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.233325 15359 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.265391 15461 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:46.226835 15382 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.255298 15129 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:16:46.279471 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.217995 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.258852 11472 raft_consensus.cc:2410] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:16:46.266844 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 74144.3us
I20260709 13:16:46.220877 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.273895 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:46.209259 11475 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.225297 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.282392 11468 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11.
I20260709 13:16:46.282608 15565 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:46.282698 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.283243 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.283353 15552 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.283778 11397 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.283918 15566 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.254822 15573 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:46.284590 15566 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.284794 15573 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.285111 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.286711 15566 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.277304 11548 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:46.211807 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.278469 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.221177 15354 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:46.279572 15359 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:46.202711 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:46.271612 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 127847us
I20260709 13:16:46.303050 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.222833 12119 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:46.287973 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.284736 12116 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:46.292975 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.293254 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.293428 11393 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:16:46.292475 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.293926 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.205713 15421 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:46.294373 11477 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:46.312098 11468 raft_consensus.cc:2393] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:46.313218 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.283998 15552 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.283996 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.295995 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.316954 11402 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:46.287550 15309 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:46.287690 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 88765.5us
I20260709 13:16:46.289976 15452 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:46.268173 15450 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.287770 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.299645 11476 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:46.301864 11397 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:46.316228 11472 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:46.318219 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.325990 11547 raft_consensus.cc:2410] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:46.323630 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.203727 15562 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:46.326560 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2355) took 1130 ms. Trace:
I20260709 13:16:46.311883 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:46.207669 15634 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:46.332162 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44308.8us
I20260709 13:16:46.332329 15438 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.317023 11395 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:46.332952 15634 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:46.333630 15634 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:46.333961 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.334420 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 2042) took 1619 ms. Trace:
I20260709 13:16:46.334730 15634 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.334846 11550 rpcz_store.cc:276] 0709 13:16:44.715386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.325178 (+1609792us) service_pool.cc:224] Handling call
0709 13:16:46.333699 (+  8521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.334070 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.334112 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.334152 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:46.334180 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.334239 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.334347 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1101,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79}
I20260709 13:16:46.336514 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.336671 15636 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.328315 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:46.337275 15636 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.337747 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.338514 11550 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:46.330267 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.322542 15382 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:46.339712 11399 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:46.340062 15382 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.340595 15382 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.322343 15565 raft_consensus.cc:697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:46.330515 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.346132 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.327726 15638 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:46.331422 15327 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.318261 15474 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:46.347028 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 62008.7us
I20260709 13:16:46.347898 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.349733 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.352039 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:46.355495 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51324.8us
I20260709 13:16:46.326781 11470 raft_consensus.cc:2410] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:46.364280 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:46.286832 15573 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.330763 12123 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:46.311291 15566 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.397480 11545 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:46.417932 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2356) took 1221 ms. Trace:
I20260709 13:16:46.417829 15566 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.418325 11545 rpcz_store.cc:276] 0709 13:16:45.196576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.192748 (+996172us) service_pool.cc:224] Handling call
0709 13:16:46.351845 (+159097us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:46.417874 (+ 66029us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177102208}
I20260709 13:16:46.419631 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.420113 15566 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.343106 15565 consensus_queue.cc:237] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.420806 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.422447 15552 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.422647 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.423183 11545 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.423511 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.386644 11475 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:46.425544 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.425997 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:46.426285 11475 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:46.426796 15473 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:46.427085 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.427448 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.389155 11402 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:46.391839 11546 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:46.328068 11547 rpcz_store.cc:276] 0709 13:16:45.196218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.189252 (+993034us) service_pool.cc:224] Handling call
0709 13:16:46.326509 (+137257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.428802 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2352) took 1233 ms. Trace:
I20260709 13:16:46.428866 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.429214 11546 rpcz_store.cc:276] 0709 13:16:45.195051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.262051 (+1067000us) service_pool.cc:224] Handling call
0709 13:16:46.345816 (+ 83765us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b5800120398. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8df0a0 00007f8dae8de984 00007f8da7e596d4 00007f8da7e59662
0709 13:16:46.428746 (+ 82930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58250496}
W20260709 13:16:46.393950 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 76673.7us
I20260709 13:16:46.332376 15354 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.398317 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2345) took 1250 ms. Trace:
I20260709 13:16:46.356977 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.400014 11543 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 4: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:46.400584 11477 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:46.400708 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.401530 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:46.398702 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:46.403348 15327 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:46.403728 15466 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:46.403879 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:46.405143 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:46.405560 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:46.407074 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.407397 15474 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:46.408228 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.448294 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:46.448565 11470 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:46.409415 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.409863 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.449620 11542 raft_consensus.cc:2393] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:46.393399 15436 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:46.411942 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.333099 15421 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.427774 11475 raft_consensus.cc:2393] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:46.428254 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.429520 11547 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:46.325496 15146 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:46.463506 15146 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.463701 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2343) took 1327 ms. Trace:
I20260709 13:16:46.463963 15146 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.464049 11547 rpcz_store.cc:276] 0709 13:16:45.136119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.428936 (+1292817us) service_pool.cc:224] Handling call
0709 13:16:46.429230 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.463412 (+ 34182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.463617 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.465416 11547 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.465971 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2349) took 1272 ms. Trace:
I20260709 13:16:46.466266 11547 rpcz_store.cc:276] 0709 13:16:45.193716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.464936 (+1271220us) service_pool.cc:224] Handling call
0709 13:16:46.465258 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.465752 (+   494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.465900 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.402170 11396 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.431912 11551 rpcz_store.cc:276] 0709 13:16:45.136937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.614731 (+477794us) service_pool.cc:224] Handling call
0709 13:16:46.356408 (+741677us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400404554. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:46.387641 (+ 31233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755000064}
I20260709 13:16:46.437412 11549 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:46.390821 11474 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:46.459013 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1658) took 2190 ms (client timeout 2260 ms). Trace:
I20260709 13:16:46.442178 12117 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:46.448868 11399 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.411229 11472 raft_consensus.cc:2410] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:16:46.333009 15438 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.459520 12119 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:46.458396 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 49074.7us
I20260709 13:16:46.475258 15509 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:46.475489 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.430728 15354 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.459067 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:46.478730 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:46.449796 15452 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:46.458581 15636 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:46.476850 12122 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.458600 15097 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:46.471887 11400 rpcz_store.cc:269] 0709 13:16:44.259273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.260312 (+  1039us) service_pool.cc:224] Handling call
0709 13:16:46.353798 (+2093486us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540030b954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:46.449301 (+ 95503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3867729664}
I20260709 13:16:46.442626 15129 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:46.480935 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.490944 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:46.437978 11545 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:46.476200 15566 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:46.494426 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2358) took 1297 ms. Trace:
I20260709 13:16:46.491012 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:46.495035 11547 rpcz_store.cc:276] 0709 13:16:45.197277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.466908 (+1269631us) service_pool.cc:224] Handling call
0709 13:16:46.469444 (+  2536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.494049 (+ 24605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.494083 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.494127 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.494156 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.494215 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.494359 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.495844 11470 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:46.476306 15436 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:46.503365 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2359) took 1305 ms. Trace:
I20260709 13:16:46.476475 15509 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:46.509851 11474 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:46.503762 11543 rpcz_store.cc:276] 0709 13:16:45.197634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.469837 (+1272203us) service_pool.cc:224] Handling call
0709 13:16:46.491570 (+ 21733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.497054 (+  5484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.497081 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.503081 (+  6000us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:46.503132 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.503202 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.503309 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:46.509730 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 81606.7us
I20260709 13:16:46.502107 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:46.509749 11399 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:46.491226 15097 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.519021 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2362) took 1320 ms. Trace:
I20260709 13:16:46.521489 11547 rpcz_store.cc:276] 0709 13:16:45.198556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.499676 (+1301120us) service_pool.cc:224] Handling call
0709 13:16:46.518396 (+ 18720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.518601 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.518638 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.518691 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:46.518733 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.518803 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.518932 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.520382 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:46.490299 15461 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:16:46.496807 15359 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:46.536759 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:46.537149 11514 leader_election.cc:400] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:46.537518 11514 leader_election.cc:403] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:46.537438 11475 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:46.539724 11548 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:16:46.503021 15129 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:46.512210 15523 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:46.513249 15636 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:46.545378 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:46.540129 11548 pending_rounds.cc:85] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 8
I20260709 13:16:46.550182 11548 pending_rounds.cc:107] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:16:46.550568 11548 raft_consensus.cc:2889] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:46.558629 12122 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:46.524081 15097 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.565145 11543 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:46.523720 15452 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:46.568926 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:46.573342 11514 leader_election.cc:400] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:16:46.547502 15461 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:46.576709 15438 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:46.578631 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85682.4us
I20260709 13:16:46.580138 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:46.536286 15473 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.583094 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2364) took 1377 ms. Trace:
I20260709 13:16:46.584278 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 2049) took 1178 ms. Trace:
I20260709 13:16:46.584643 11543 rpcz_store.cc:276] 0709 13:16:45.407240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.537993 (+1130753us) service_pool.cc:224] Handling call
0709 13:16:46.564954 (+ 26961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.584031 (+ 19077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.584229 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.583503 11548 rpcz_store.cc:276] 0709 13:16:45.206548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.539110 (+1332562us) service_pool.cc:224] Handling call
0709 13:16:46.539401 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.582813 (+ 43412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.583034 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.591305 15630 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:16:46.593401 15509 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.594944 15509 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:46.596460 15453 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:46.596956 15453 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.597414 15453 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:46.599072 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:46.599788 11544 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.600467 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 2054) took 1079 ms. Trace:
I20260709 13:16:46.600854 11549 rpcz_store.cc:276] 0709 13:16:45.521254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.002569 (+481315us) service_pool.cc:224] Handling call
0709 13:16:46.600302 (+597733us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54002071d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:46.600410 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702512896}
W20260709 13:16:46.603053 15505 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:16:46.604100 11550 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.604979 15466 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:46.605800 11550 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:46.564437 15636 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.611420 15636 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:46.621342 15538 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:46.620642 11299 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:46.621891 15538 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:46.623409 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78114.1us
I20260709 13:16:46.623660 15516 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:46.625453 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.625752 15516 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:46.570946 15452 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:46.627168 15452 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.628553 15436 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:46.623440 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47810.6us
W20260709 13:16:46.631788 15645 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:16:46.668144 11547 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:46.680172 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 208295us
I20260709 13:16:46.680819 11543 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:46.682152 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 65471.2us
I20260709 13:16:46.689615 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:46.693549 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 2051) took 1226 ms. Trace:
I20260709 13:16:46.694114 11547 rpcz_store.cc:276] 0709 13:16:45.459317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.593836 (+1134519us) service_pool.cc:224] Handling call
0709 13:16:46.594156 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.668675 (+ 74519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.683749 (+ 15074us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b540033d1d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:46.683990 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169683200}
I20260709 13:16:46.696519 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2366) took 1232 ms. Trace:
W20260709 13:16:46.688416 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 132965us
W20260709 13:16:46.697345 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 213307us
I20260709 13:16:46.697095 11543 rpcz_store.cc:276] 0709 13:16:45.456073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.585760 (+1129687us) service_pool.cc:224] Handling call
0709 13:16:46.680606 (+ 94846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.687715 (+  7109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.687982 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.681608 15526 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:46.677350 15474 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:46.731962 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:46.736213 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 156194us
I20260709 13:16:46.738729 11550 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:46.742540 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:46.738857 11516 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:46.744767 11547 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:46.700599 15450 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.747462 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 111982us
I20260709 13:16:46.692205 15248 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:16:46.758765 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69248.2us
I20260709 13:16:47.041689 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:46.740782 15645 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 13:16:46.799834 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61361.4us
I20260709 13:16:46.803120 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 624) took 2118 ms. Trace:
I20260709 13:16:46.752849 15474 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:46.809348 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:46.815412 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 130369us
I20260709 13:16:47.048377 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.029269 11544 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:16:46.800556 15177 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:46.807087 15129 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.696575 15382 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:46.744910 15573 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:47.030476 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:46.684369 15473 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:46.633538 15538 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.061945 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:47.033236 12119 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.006s
I20260709 13:16:46.799535 15080 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:16:47.032439 15590 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:47.043181 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 547) took 1210 ms. Trace:
I20260709 13:16:47.084619 11548 rpcz_store.cc:276] 0709 13:16:45.819181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.704729 (+885548us) service_pool.cc:224] Handling call
0709 13:16:46.740445 (+ 35716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.759928 (+ 19483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.759953 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.029697 (+269744us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:47.029752 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.029810 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.029930 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.043777 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 548) took 1026 ms. Trace:
I20260709 13:16:47.047230 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:47.066733 15646 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 13:16:47.071996 12119 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:46.739554 15526 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:47.078006 15633 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:47.078553 15538 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:47.079244 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.080001 15565 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:47.081650 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1623) took 1038 ms. Trace:
I20260709 13:16:47.091573 11550 rpcz_store.cc:276] 0709 13:16:46.010214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.696873 (+686659us) service_pool.cc:224] Handling call
0709 13:16:46.735341 (+ 38468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.760553 (+ 25212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.036281 (+275728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:47.092857 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:47.050882 15505 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:47.046867 12115 rpcz_store.cc:276] 0709 13:16:44.584218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.613341 (+ 29123us) service_pool.cc:224] Handling call
0709 13:16:44.613650 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.613854 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.613877 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.260752 (+1646875us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b2c000476c8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:46.261175 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.459738 (+198563us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:46.463261 (+  3523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.463392 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.696999 (+233607us) raft_consensus.cc:1685] finished
0709 13:16:46.697020 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.702659 (+  5639us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6550400,"thread_start_us":198216,"threads_started":1,"wal-append.queue_time_us":309915}
I20260709 13:16:47.050755 15516 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:47.103035 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56060.6us
I20260709 13:16:47.106824 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:47.108822 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 366372us
I20260709 13:16:47.070703 15354 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:47.073441 15177 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:47.084148 15080 consensus_queue.cc:1243] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:16:47.114782 15644 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.357s	user 0.004s	sys 0.006s
I20260709 13:16:47.110934 15526 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:47.046460 15634 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:16:47.111935 11469 rpcz_store.cc:276] 0709 13:16:46.043307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.065081 (+ 21774us) service_pool.cc:224] Handling call
0709 13:16:46.200969 (+135888us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b5800023498. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f4a22 00007f8daf70dc09 00007f8dae867ee0 00007f8dae9310ab 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:46.201238 (+   269us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:46.255414 (+ 54176us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:46.271891 (+ 16477us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.584279 (+312388us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:46.602135 (+ 17856us) log.cc:844] Serialized 52 byte log entry
0709 13:16:46.602299 (+   164us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:46.602815 (+   516us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.605310 (+  2495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.605376 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.081412 (+476036us) raft_consensus.cc:1685] finished
0709 13:16:47.081434 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.081577 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2052,"prepare.queue_time_us":4525,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":83357,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":195,"spinlock_wait_cycles":15175936,"thread_start_us":312041,"threads_started":1,"wal-append.queue_time_us":794893}
I20260709 13:16:47.111130 15633 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.201372 11399 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:16:47.203109 11546 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.020s	sys 0.034s
I20260709 13:16:47.201753 11396 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:47.203907 11546 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.020s	sys 0.035s
I20260709 13:16:47.058838 15450 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.129361 15523 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:47.144826 15516 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:47.113981 15590 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:47.216164 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 166609us
W20260709 13:16:47.197845 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 167431us
I20260709 13:16:47.225764 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.234560 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:47.215751 15633 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:47.279021 11470 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.774s	user 0.008s	sys 0.004s
I20260709 13:16:47.221940 15562 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:47.233464 15516 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:47.240293 15634 consensus_queue.cc:1243] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:47.282251 11470 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:47.240095 15641 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:47.154281 15573 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.250366 15523 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:47.234169 15566 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:47.327683 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 582) took 1244 ms. Trace:
I20260709 13:16:47.333449 15528 raft_consensus.cc:2764] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:16:47.290088 15562 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.333649 15566 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.335451 15528 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:47.335966 15528 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:47.336071 15562 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:47.336648 15566 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:47.299669 15466 raft_consensus.cc:2764] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:16:47.336990 15528 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:47.339442 15647 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
W20260709 13:16:47.340312 15650 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 13:16:47.342265 15452 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.000s
W20260709 13:16:47.343099 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 127877us
I20260709 13:16:47.355788 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43442 (request call id 2052) took 1842 ms. Trace:
I20260709 13:16:47.356285 11549 rpcz_store.cc:276] 0709 13:16:45.515074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.601625 (+1086551us) service_pool.cc:224] Handling call
0709 13:16:46.601938 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.632983 (+ 31045us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:46.633027 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.667985 (+ 34958us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.155618 (+487633us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:47.155657 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.155720 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.355553 (+199833us) raft_consensus.cc:1685] finished
0709 13:16:47.355582 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.355732 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30623,"raft.queue_time_us":51170,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":8993,"thread_start_us":424063,"threads_started":1,"wal-append.queue_time_us":609281}
W20260709 13:16:47.330214 15636 consensus_meta.cc:287] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.713s	user 0.015s	sys 0.004s
I20260709 13:16:47.357985 15636 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.358686 11549 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:47.355367 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2361) took 2157 ms. Trace:
I20260709 13:16:47.359205 15573 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:47.359928 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1618) took 1216 ms. Trace:
I20260709 13:16:47.360194 15636 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:47.357017 15590 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.364187 15450 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:47.352834 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 273722us
W20260709 13:16:47.370360 15649 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 13:16:47.371692 15661 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:16:47.372491 15473 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:47.328240 11401 rpcz_store.cc:276] 0709 13:16:46.074864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.075058 (+   194us) service_pool.cc:224] Handling call
0709 13:16:46.075398 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.117375 (+ 41977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.117419 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.122733 (+  5314us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.207310 (+ 84577us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b18003fe800. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:46.512041 (+304731us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.512411 (+   370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.512478 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.302235 (+789757us) raft_consensus.cc:1685] finished
0709 13:16:47.302264 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.319103 (+ 16839us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28562,"raft.queue_time_us":36659,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5079,"spinlock_wait_cycles":3365632,"thread_start_us":300310,"threads_started":1,"wal-append.queue_time_us":561327}
I20260709 13:16:47.378324 11545 rpcz_store.cc:276] 0709 13:16:45.198268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.493345 (+1295077us) service_pool.cc:224] Handling call
0709 13:16:46.493638 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.493952 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.493979 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.518625 (+ 24646us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.693957 (+175332us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:46.694299 (+   342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.694383 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.355090 (+660707us) raft_consensus.cc:1685] finished
0709 13:16:47.355116 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.355295 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":401,"raft.queue_time_us":99,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3981,"thread_start_us":129084,"threads_started":1,"wal-append.queue_time_us":677748}
W20260709 13:16:47.345362 15146 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.881s	user 0.009s	sys 0.001s
I20260709 13:16:47.412619 15146 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.413766 11543 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:47.333787 15523 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.415802 15146 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:47.351161 15382 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:47.411984 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.426188 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:47.403332 15644 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:16:47.342672 15452 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.467154 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2370) took 1131 ms. Trace:
I20260709 13:16:47.467630 11547 rpcz_store.cc:276] 0709 13:16:46.336088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.753344 (+417256us) service_pool.cc:224] Handling call
0709 13:16:47.041662 (+288318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.064729 (+ 23067us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:47.111655 (+ 46926us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.466735 (+355080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.466796 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:47.466841 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.466914 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.467083 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151922,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":24301,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":280008,"threads_started":1}
I20260709 13:16:47.381481 11473 rpcz_store.cc:276] 0709 13:16:46.004213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.057416 (+ 53203us) service_pool.cc:224] Handling call
0709 13:16:46.074314 (+ 16898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.084642 (+ 10328us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:46.123498 (+ 38856us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.308646 (+185148us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.326137 (+ 17491us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.326275 (+   138us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.326306 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.326366 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.220530 (+894164us) raft_consensus.cc:1685] finished
0709 13:16:47.220567 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.220714 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51590,"prepare.queue_time_us":27124,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":6246,"raft.queue_time_us":39940,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":8828,"thread_start_us":123755,"threads_started":1,"wal-append.queue_time_us":328906}
I20260709 13:16:47.303037 15177 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.443025 15636 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:47.444804 15382 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:47.499549 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 87640.8us
W20260709 13:16:47.502087 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97563.4us
I20260709 13:16:47.503001 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.507563 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:47.469224 15516 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.513397 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:47.514758 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.513922 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1621) took 1507 ms. Trace:
W20260709 13:16:47.515542 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 273391us
I20260709 13:16:47.519510 12083 leader_election.cc:400] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11.
I20260709 13:16:47.518296 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2344) took 2381 ms. Trace:
W20260709 13:16:47.508828 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 239699us
W20260709 13:16:47.542528 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.521983 11546 rpcz_store.cc:276] 0709 13:16:45.136566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.430041 (+1293475us) service_pool.cc:224] Handling call
0709 13:16:46.430325 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.430568 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:47.220431 (+789863us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.516377 (+295946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.516449 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:47.516511 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.518078 (+  1567us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.518227 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619080,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":9579,"thread_start_us":996672,"threads_started":2}
I20260709 13:16:47.547894 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:47.517176 15382 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:47.556623 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.558738 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 706) took 1243 ms. Trace:
I20260709 13:16:47.515668 11471 rpcz_store.cc:276] 0709 13:16:46.006955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.015157 (+  8202us) service_pool.cc:224] Handling call
0709 13:16:46.015564 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:46.017402 (+  1838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.017435 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:46.017996 (+   561us) log.cc:844] Serialized 52 byte log entry
0709 13:16:46.434403 (+416407us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:46.476803 (+ 42400us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.504379 (+ 27576us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.504638 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.504713 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.513682 (+1008969us) raft_consensus.cc:1685] finished
0709 13:16:47.513707 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.513861 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46819,"prepare.queue_time_us":108365,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":1392,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":17985,"thread_start_us":415952,"threads_started":1,"wal-append.queue_time_us":614397}
I20260709 13:16:47.558190 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:47.559188 11476 rpcz_store.cc:276] 0709 13:16:46.307979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.428636 (+120657us) service_pool.cc:224] Handling call
0709 13:16:47.551685 (+1123049us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b540044b1d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:47.551802 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525467904}
I20260709 13:16:47.563081 15505 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:47.546968 15080 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:16:47.549062 11441 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.793s (SENT) [suppressed 6 similar messages]
I20260709 13:16:47.549685 15636 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:47.551110 15453 consensus_meta.cc:287] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.953s	user 0.013s	sys 0.000s
I20260709 13:16:47.551398 15473 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:47.573875 15452 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:47.580395 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 549) took 1554 ms. Trace:
W20260709 13:16:47.591450 15664 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:16:47.603901 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 54876.8us
I20260709 13:16:47.558842 15382 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:47.605252 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 58154.1us
I20260709 13:16:47.542264 15667 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:47.566864 15450 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:47.608824 11543 rpcz_store.cc:276] 0709 13:16:46.026403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.700105 (+673702us) service_pool.cc:224] Handling call
0709 13:16:46.735432 (+ 35327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.414188 (+678756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.580076 (+165888us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b5400134cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:47.580311 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559631744}
I20260709 13:16:47.609861 15080 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.618795 15450 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:47.608518 15453 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.605091 15636 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:47.601338 15674 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:47.570829 15670 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:47.604604 15509 consensus_meta.cc:287] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.009s	user 0.009s	sys 0.004s
W20260709 13:16:47.603407 15097 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.017s	user 0.011s	sys 0.000s
I20260709 13:16:47.617787 15382 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:47.614955 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 56883.6us
I20260709 13:16:47.608829 15573 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:16:47.627600 11542 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:47.617842 15668 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:47.622526 15636 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:47.628820 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.012s
W20260709 13:16:47.632468 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.008s	sys 0.013s
W20260709 13:16:47.633625 11551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.004s	sys 0.016s
W20260709 13:16:47.635809 11551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.004s	sys 0.016s
I20260709 13:16:47.638060 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.642513 15671 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:47.644747 11546 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:47.646282 11547 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:47.649727 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:47.650324 11544 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:47.651940 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.625911 15097 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.634229 15668 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:47.623770 15526 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.663748 11394 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:47.666357 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 121466us
I20260709 13:16:47.640239 15516 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:47.668323 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.667949 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:47.670910 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.649792 15474 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:47.677631 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:47.672631 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.680253 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.680435 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:47.635139 15675 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:16:47.664710 15354 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.711769 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:47.664738 15452 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.666004 15670 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:47.762197 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:47.749606 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:47.763814 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 211832us
I20260709 13:16:47.767117 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:47.684116 11394 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:16:47.768998 11394 pending_rounds.cc:85] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 10
I20260709 13:16:47.769761 11394 pending_rounds.cc:107] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:16:47.770213 11394 raft_consensus.cc:2889] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:47.752071 11396 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:47.674310 15248 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:16:47.772517 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:47.773142 12122 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:47.752578 11544 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:47.754577 11397 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:47.660741 15630 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:16:47.659880 15674 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:16:47.750767 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77235.9us
I20260709 13:16:47.748441 15474 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:47.771073 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.782016 12116 pending_rounds.cc:85] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 9
I20260709 13:16:47.782321 12116 pending_rounds.cc:107] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:16:47.782590 12116 raft_consensus.cc:2889] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:47.774199 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:47.778218 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:47.780112 11402 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:47.780296 12119 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:47.780964 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.707811 15473 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:16:47.777173 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:47.708493 15436 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:16:47.758780 15677 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:47.806454 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:47.755388 15523 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:16:47.676465 15450 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:47.788057 15593 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:47.786872 15474 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:47.873737 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:47.860688 12122 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:47.861919 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:47.765563 15590 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:16:47.763415 15354 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:47.817188 15633 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.002s
W20260709 13:16:47.884932 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 70852.2us
W20260709 13:16:47.861675 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55277.1us
I20260709 13:16:47.809049 15080 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:47.816890 15674 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.866227 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:47.864780 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:47.777334 15452 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:47.868471 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 83451.6us
I20260709 13:16:47.804642 15634 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:16:47.864939 15177 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:16:47.864812 15631 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.010s	sys 0.006s
I20260709 13:16:47.889298 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:47.884032 15538 consensus_meta.cc:287] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.000s
I20260709 13:16:47.863315 15453 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:47.867969 15474 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:47.855103 15552 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:47.884575 15518 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:47.983639 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 215160us
I20260709 13:16:47.816154 15688 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:16:47.868325 15327 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:47.863999 15691 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:47.888239 15436 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:47.884932 15421 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.930879 15675 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:48.008908 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:47.996762 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:47.818773 15677 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.850924 15668 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.010114 11548 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:47.992931 15097 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:47.999382 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 84433.2us
W20260709 13:16:47.990644 15631 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.011s	sys 0.006s
I20260709 13:16:48.004837 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:48.005239 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:47.913339 15633 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:47.991658 15566 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:48.001299 15080 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:47.999100 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 287390us
I20260709 13:16:48.009166 15675 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:48.032934 11470 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:16:48.033350 11470 pending_rounds.cc:85] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 4
I20260709 13:16:48.033655 11470 pending_rounds.cc:107] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:16:48.033957 11470 raft_consensus.cc:2889] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:48.010164 15691 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.010296 15436 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:48.003820 15526 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:48.030715 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.040190 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:48.042716 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.026678 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:48.034742 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.044507 11477 raft_consensus.cc:2410] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:16:48.001703 15248 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.983161 15674 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.033368 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.054666 11469 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:48.014853 15416 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:48.037004 11397 raft_consensus.cc:2368] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:48.037101 11543 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:48.045504 11548 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:48.054571 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:48.041625 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.043329 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:48.080324 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.036670 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:48.035049 15680 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:16:48.081965 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.089457 11477 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:48.080974 11470 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:48.013140 15552 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:48.049796 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 292631us
W20260709 13:16:48.117349 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 62902us
W20260709 13:16:48.091187 15690 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:16:48.103714 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 100201us
I20260709 13:16:48.091826 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:48.104982 11468 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:48.036586 15675 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.036697 15638 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:48.124430 15676 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:16:48.125036 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:48.125437 15689 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 13:16:48.126012 15684 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 13:16:48.126416 15698 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:48.127595 11476 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:48.127679 15423 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.128296 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:48.128535 15687 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:16:48.128922 15697 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.129025 15509 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:48.129477 15682 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.348s	user 0.000s	sys 0.002s
I20260709 13:16:48.133520 15552 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.028065 15327 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.135456 15474 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.141597 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2363) took 2942 ms. Trace:
I20260709 13:16:48.259724 11542 rpcz_store.cc:276] 0709 13:16:45.198855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.539013 (+1340158us) service_pool.cc:224] Handling call
0709 13:16:46.562382 (+ 23369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.628030 (+1065648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.141266 (+513236us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400269654. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:48.141513 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2449387264}
I20260709 13:16:48.143852 15630 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:48.130738 15670 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:48.146195 11543 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:48.156841 11477 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:48.015532 15421 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.092167 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.164520 15177 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:48.116909 15678 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:16:48.091578 15679 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:16:48.038446 15691 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:48.146710 15685 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 13:16:48.042080 15566 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.210672 11548 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:48.168820 11469 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:48.212620 11471 raft_consensus.cc:2410] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:16:48.210202 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:48.293730 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 83591.4us
I20260709 13:16:48.295246 12122 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:48.221496 11475 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:48.090127 15248 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.224126 15703 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:48.224884 15146 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.225701 15677 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.272161 15354 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:48.226352 15452 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.226506 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2375) took 1189 ms. Trace:
I20260709 13:16:48.227218 15453 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.309755 11543 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:48.250290 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:48.106043 15080 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.211596 15683 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:48.266395 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.266660 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:48.276162 11470 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
I20260709 13:16:48.159220 11547 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:48.217113 15638 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.285679 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.253073 15698 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.263826 15630 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:48.303395 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.268939 15509 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:48.267102 15670 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:48.279310 15528 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.310266 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:48.310561 11549 rpcz_store.cc:276] 0709 13:16:47.036636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.417737 (+381101us) service_pool.cc:224] Handling call
0709 13:16:47.418069 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.418427 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.418453 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.441714 (+ 23261us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.751723 (+310009us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:47.763378 (+ 11655us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.763548 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.763630 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.127224 (+363594us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b180039d9c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1
0709 13:16:48.128927 (+  1703us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b180039d9c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:48.226291 (+ 97364us) raft_consensus.cc:1685] finished
0709 13:16:48.226314 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.226445 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":654,"prepare.queue_time_us":11523,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":21081,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":41122,"spinlock_wait_cycles":36040704,"thread_start_us":284098,"threads_started":1,"wal-append.queue_time_us":338096}
I20260709 13:16:48.316201 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.180910 15436 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.370791 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.371426 11469 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
W20260709 13:16:48.319557 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 69347.6us
I20260709 13:16:48.320958 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.322306 11545 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:48.265625 15423 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.366159 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:48.370141 12112 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:16:48.370226 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.383755 11400 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:48.384377 11476 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:48.316237 15452 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.380829 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:48.367424 15699 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:16:48.399716 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.402410 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:48.407163 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97007.6us
I20260709 13:16:48.285560 15697 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.386057 11468 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:16:48.365283 15677 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:48.309412 15146 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:48.417706 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44927.6us
I20260709 13:16:48.364620 15638 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.423822 15702 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:48.423938 15696 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:48.382282 15633 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.410432 15528 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.425153 15721 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.425640 15713 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:16:48.425856 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:48.426030 15658 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:48.368326 15683 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:48.439700 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2372) took 2103 ms. Trace:
I20260709 13:16:48.321518 15552 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:16:48.444159 11545 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:48.485608 11550 rpcz_store.cc:276] 0709 13:16:46.336605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.142787 (+806182us) service_pool.cc:224] Handling call
0709 13:16:47.143380 (+   593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.197174 (+ 53794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.197205 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.270158 (+ 72953us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.553127 (+282969us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:47.553178 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.553270 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.439472 (+886202us) raft_consensus.cc:1685] finished
0709 13:16:48.439497 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.439640 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11731,"raft.queue_time_us":53923,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":14950,"thread_start_us":233202,"threads_started":1,"wal-append.queue_time_us":354911}
I20260709 13:16:48.486819 11545 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:48.321377 15453 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:48.449565 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 236838us
I20260709 13:16:48.463665 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.468500 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.493753 12117 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:48.470816 11543 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:48.472110 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.496611 12118 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:48.480719 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.497061 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:48.415763 15473 consensus_queue.cc:1175] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "147201d22a694ca59f6e014f083a0892" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
I20260709 13:16:48.512833 11475 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:48.502027 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.503355 15248 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.413440 15452 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.505493 15453 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:48.485217 15436 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:48.507582 15552 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:48.501998 15421 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.513618 11476 raft_consensus.cc:2468] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:48.521104 12113 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:16:48.521530 11469 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:48.481441 15658 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:48.527750 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.470211 15526 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:16:48.538611 11476 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:48.567492 15702 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:48.570425 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.429095 15177 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.588011 12118 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:48.464910 15696 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.592932 11395 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:48.577283 15327 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.703161 15702 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.571583 15509 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.667179 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.707662 12113 raft_consensus.cc:2410] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:16:48.707931 12117 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:48.578390 15453 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.131s
I20260709 13:16:48.578706 15474 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:48.711957 15720 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:16:48.704258 11395 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:16:48.670014 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:48.678763 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:48.718724 11475 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:48.574011 15538 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:48.491005 15721 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.757292 11474 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:48.750885 15667 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:48.483026 15630 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.762544 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.762794 15453 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:48.765475 15528 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.766083 15696 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.796913 11476 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.800585 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.774852 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.793674 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:48.585278 15674 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.795308 15453 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.767256 15566 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.768266 15691 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.885915 11470 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:48.856478 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.855844 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.896643 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:48.898299 11395 raft_consensus.cc:2410] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:48.843048 15630 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:48.887887 15723 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
W20260709 13:16:48.888322 15730 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:16:48.885146 15453 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:48.888892 15724 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:16:48.915103 15708 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.518s	user 0.001s	sys 0.001s
W20260709 13:16:48.917194 15728 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:48.918929 12116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 783) took 1160 ms. Trace:
W20260709 13:16:48.896633 15722 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:16:48.919498 12116 rpcz_store.cc:276] 0709 13:16:47.758751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.758868 (+   117us) service_pool.cc:224] Handling call
0709 13:16:47.781813 (+ 22945us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:47.862511 (+ 80698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.862544 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:47.870728 (+  8184us) log.cc:844] Serialized 52 byte log entry
0709 13:16:48.158610 (+287882us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:48.158647 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.158713 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.918658 (+759945us) raft_consensus.cc:1685] finished
0709 13:16:48.918681 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.918850 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15043,"raft.queue_time_us":74852,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":66066,"spinlock_wait_cycles":1582976,"thread_start_us":170952,"threads_started":1,"wal-append.queue_time_us":376858}
I20260709 13:16:48.921572 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2382) took 1034 ms. Trace:
I20260709 13:16:48.900563 15248 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:48.921911 11542 rpcz_store.cc:276] 0709 13:16:47.886813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.283349 (+396536us) service_pool.cc:224] Handling call
0709 13:16:48.300590 (+ 17241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:48.307072 (+  6482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.307105 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:48.307540 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:16:48.526371 (+218831us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:48.562859 (+ 36488us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.571700 (+  8841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.571772 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.921364 (+349592us) raft_consensus.cc:1685] finished
0709 13:16:48.921385 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.921515 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73326,"prepare.queue_time_us":41395,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":72683,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":35869,"thread_start_us":156166,"threads_started":1,"wal-append.queue_time_us":245710}
I20260709 13:16:48.923561 15248 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.760015 15667 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:48.919049 15737 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:48.925289 15667 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.926175 15248 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.926362 15667 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:48.900807 15421 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:48.927275 15421 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.929148 15421 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.903956 15423 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:48.914245 15382 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 1.284s	user 0.012s	sys 0.000s
I20260709 13:16:48.937563 15382 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.938247 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.756022 15658 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.939201 15638 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.939705 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.940744 15658 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:48.941804 15638 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.941895 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.896010 15080 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.943676 15739 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:48.944054 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.944232 15739 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.944787 12119 raft_consensus.cc:2368] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:48.944974 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2360) took 3747 ms. Trace:
I20260709 13:16:48.946393 15739 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.947132 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 782) took 1196 ms. Trace:
I20260709 13:16:48.947495 12115 rpcz_store.cc:276] 0709 13:16:47.751022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.751167 (+   145us) service_pool.cc:224] Handling call
0709 13:16:47.758717 (+  7550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.032438 (+273721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.032472 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.032510 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:48.032541 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.946898 (+914357us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54003778d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:48.946945 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.947079 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629072640}
I20260709 13:16:48.948532 15721 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:48.949224 11476 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:48.767374 15538 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:48.939597 15382 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:48.951402 11471 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:48.911574 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:48.943672 12118 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:48.945367 11551 rpcz_store.cc:276] 0709 13:16:45.197954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.473042 (+1275088us) service_pool.cc:224] Handling call
0709 13:16:46.484610 (+ 11568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.521445 (+ 36835us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:47.033837 (+512392us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.656011 (+622174us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.662571 (+  6560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.662639 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:47.662696 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.764498 (+1101802us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b480002d9d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:48.944759 (+180261us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54002c60d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:48.944809 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.944923 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72803,"prepare.queue_time_us":612450,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":120,"raft.queue_time_us":33985,"raft.run_cpu_time_us":3531,"raft.run_wall_time_us":1104682,"spinlock_wait_cycles":35752576,"thread_start_us":1025580,"threads_started":2,"wal-append.queue_time_us":813098}
I20260709 13:16:48.678587 15683 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:48.912375 15698 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:48.978055 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.980798 12119 raft_consensus.cc:2393] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:48.983223 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:48.987490 12118 raft_consensus.cc:2368] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:48.999250 15566 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:48.999821 15566 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.000164 15698 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:49.001346 15566 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.001778 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:16:49.003346 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.004575 15691 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.006397 12118 raft_consensus.cc:2410] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:49.006443 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:49.007743 11515 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.008903 15698 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:49.011641 11473 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:48.978754 15452 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:49.010908 15675 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.781s	user 0.004s	sys 0.004s
I20260709 13:16:49.003537 15538 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.033051 15698 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.033862 15698 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.044010 11441 leader_election.cc:400] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:49.044538 11441 leader_election.cc:403] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.047369 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.047375 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:49.057376 12083 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:49.048043 15674 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:49.027036 15675 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.058725 15698 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.022874 15701 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:16:49.060720 15675 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.061619 15666 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:49.062165 15666 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:49.062583 15666 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:49.064493 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:49.071849 15681 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.203s	user 0.004s	sys 0.000s
W20260709 13:16:49.075930 15636 consensus_meta.cc:287] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 1.438s	user 0.012s	sys 0.000s
I20260709 13:16:49.076756 15674 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:49.076503 15636 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.080143 11441 leader_election.cc:400] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:49.049081 15452 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.078598 15566 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.049187 15382 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:16:49.096139 15636 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.096311 15704 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:49.097780 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 2026) took 1333 ms. Trace:
I20260709 13:16:49.102255 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:49.107372 11397 rpcz_store.cc:276] 0709 13:16:47.765004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.816655 (+ 51651us) service_pool.cc:224] Handling call
0709 13:16:49.097569 (+1280914us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54003d40d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:49.097705 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282532608}
I20260709 13:16:49.108467 15704 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:49.093701 15700 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
W20260709 13:16:49.112603 15694 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.102s	user 0.001s	sys 0.000s
W20260709 13:16:49.094161 15593 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.005s	sys 0.001s
W20260709 13:16:49.115299 15593 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.005s	sys 0.001s
W20260709 13:16:49.117480 15436 consensus_meta.cc:287] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.000s
I20260709 13:16:49.091315 15452 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.125473 15452 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.089257 15691 raft_consensus.cc:2804] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:49.133493 11544 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:49.147297 11397 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:49.139020 15526 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:16:49.149010 15562 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:49.137771 15691 raft_consensus.cc:697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:49.149645 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.150447 15691 consensus_queue.cc:237] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.150702 15670 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.156097 15670 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:49.156265 15666 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.158631 15666 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.164122 11544 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:49.167464 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.167884 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.173154 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.173306 15450 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:49.173398 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.173823 15450 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:49.173862 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:49.174638 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.175391 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.175714 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.173166 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.177423 11474 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:49.180212 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:49.182049 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.182767 12114 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:16:49.140170 15667 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:49.183099 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.168570 12112 raft_consensus.cc:2376] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12. Re-sending same reply.
I20260709 13:16:49.182802 12123 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:49.183813 12116 raft_consensus.cc:2376] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11. Re-sending same reply.
I20260709 13:16:49.186383 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.183166 12114 pending_rounds.cc:85] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 9
I20260709 13:16:49.189738 12114 pending_rounds.cc:107] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:16:49.189971 12114 raft_consensus.cc:2889] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.188877 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.190721 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.195124 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.198283 15697 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.199472 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1639) took 1160 ms. Trace:
I20260709 13:16:49.199878 11470 rpcz_store.cc:276] 0709 13:16:48.039357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.363653 (+324296us) service_pool.cc:224] Handling call
0709 13:16:48.365952 (+  2299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.896874 (+530922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.199236 (+302362us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b5400063454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:49.199408 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190915456}
I20260709 13:16:49.174816 15450 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.207785 11474 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:49.202788 15450 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:49.193198 15675 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.211380 15675 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.190860 15562 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:49.212657 15562 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.215835 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:49.220208 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.223971 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2379) took 1101 ms. Trace:
I20260709 13:16:49.224380 11469 rpcz_store.cc:276] 0709 13:16:48.122103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.386551 (+264448us) service_pool.cc:224] Handling call
0709 13:16:48.521360 (+134809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.537267 (+ 15907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.223795 (+686528us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540039a8d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:49.223915 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300533504}
I20260709 13:16:49.225010 15562 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.232147 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:49.179289 15753 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:16:49.240454 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.239910 15636 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:16:49.237123 15505 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:49.242879 15505 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:49.243848 15505 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.245477 15505 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:49.247877 15636 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.248394 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1638) took 1210 ms. Trace:
I20260709 13:16:49.248993 15636 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.248950 11477 rpcz_store.cc:276] 0709 13:16:48.037805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.282374 (+244569us) service_pool.cc:224] Handling call
0709 13:16:48.282794 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.317392 (+ 34598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.317436 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.317489 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:48.317534 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.248071 (+930537us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5400244854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:49.248142 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.248311 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79023488}
I20260709 13:16:49.242127 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.256178 15671 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:49.256364 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.256986 12123 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:49.256759 15671 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.257335 12119 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:49.257512 15565 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:49.257920 15565 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:49.258669 12119 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:49.236884 15683 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:16:49.259701 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.259850 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.250228 15704 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.261291 15704 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:49.240108 15688 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:16:49.238654 15518 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:49.257136 12115 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:49.272684 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.273492 12116 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:49.277590 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.258865 15671 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.281585 12115 raft_consensus.cc:2368] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:49.282629 12123 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:49.293447 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:49.278914 15518 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:49.295859 15518 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.297717 11470 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:49.297879 11469 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:49.298697 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.299324 11468 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:49.299373 11475 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:49.301357 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.302148 11470 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:49.299268 11476 pending_rounds.cc:85] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 4
I20260709 13:16:49.302860 11476 pending_rounds.cc:107] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:16:49.303205 11476 raft_consensus.cc:2889] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.303143 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.310767 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.316386 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.317348 11470 raft_consensus.cc:2410] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:16:49.317533 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.320767 11546 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:49.327756 15752 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:49.328608 15667 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:49.329223 15667 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.331881 12122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 577) took 1028 ms. Trace:
I20260709 13:16:49.332372 12122 rpcz_store.cc:276] 0709 13:16:48.303727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.309073 (+  5346us) service_pool.cc:224] Handling call
0709 13:16:48.365137 (+ 56064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.469436 (+104299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.469470 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.469521 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:48.469553 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.331495 (+861942us) spinlock_profiling.cc:243] Waited 87 ms on lock 0x7b54005e1ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:49.331565 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.331763 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200191872}
I20260709 13:16:49.337430 15474 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:49.337925 15474 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:49.338792 15474 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.339877 15474 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:49.340274 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.341183 11542 raft_consensus.cc:2410] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:49.353518 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.353969 11549 pending_rounds.cc:85] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 8
I20260709 13:16:49.354302 11549 pending_rounds.cc:107] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:16:49.354573 11549 raft_consensus.cc:2889] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.306789 15636 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.357410 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:49.364529 11546 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:49.367470 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.368175 12112 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:16:49.371245 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:49.373683 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.375579 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.376278 12122 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:49.379441 11551 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:49.381270 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.381951 12123 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:49.306013 15688 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:16:49.382812 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.377817 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.359436 15703 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.385599 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.382336 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:49.395468 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.400010 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.403247 11547 raft_consensus.cc:2410] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:16:49.402460 11543 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:49.363420 15450 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.405680 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.308959 15754 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.198s
I20260709 13:16:49.409708 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.413385 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.371781 15177 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:49.340808 15327 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:16:49.420627 12119 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:49.407298 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.431497 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.433015 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.427613 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.428220 15751 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.434615 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.435197 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:49.386204 15670 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.440938 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.448457 11477 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:16:49.450169 11477 pending_rounds.cc:85] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 5
W20260709 13:16:49.474571 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.495707 11477 pending_rounds.cc:107] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:16:49.422091 15658 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.363201 15638 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
W20260709 13:16:49.499239 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 58361.9us
I20260709 13:16:49.423784 15683 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:49.493847 11547 raft_consensus.cc:1275] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:49.421291 15636 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.496033 11477 raft_consensus.cc:2889] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:49.497079 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 61964.6us
I20260709 13:16:49.530261 12114 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:49.497473 11473 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:16:49.498539 11396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1724) took 1834 ms (client timeout 2260 ms). Trace:
I20260709 13:16:49.500335 11515 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.400236 15590 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:16:49.438568 15593 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:49.515415 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 108155us
I20260709 13:16:49.434824 15080 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:49.525630 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 92690.6us
I20260709 13:16:49.538790 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.442003 15688 consensus_queue.cc:1243] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:16:49.545054 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.540321 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:49.426530 15461 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:16:49.541128 11515 leader_election.cc:400] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12.
I20260709 13:16:49.546274 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:49.532014 11396 rpcz_store.cc:269] 0709 13:16:47.663899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.664142 (+   243us) service_pool.cc:224] Handling call
0709 13:16:49.498255 (+1834113us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b540030b954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:49.498454 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12057984}
I20260709 13:16:49.543027 11441 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:49.517148 15708 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:16:49.547689 11515 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.431790 15450 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.548699 11551 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:49.486387 15452 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.549218 11441 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.439023 15701 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:16:49.524945 15683 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.537672 15097 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:49.423659 15703 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.524829 15705 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:49.527297 15726 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:49.566110 11515 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:49.544420 15698 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:49.484319 15670 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.543641 15505 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.568634 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:49.572903 11477 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:49.559881 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.573598 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:49.558379 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:49.561177 15758 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:16:49.585111 15668 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:49.560602 15688 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:49.567878 15705 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:49.563185 15566 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.607664 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:49.611145 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.559507 15725 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:49.565335 15683 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:49.567407 15097 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:49.561167 15658 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:49.556775 15674 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:49.614272 11473 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:16:49.629827 11473 pending_rounds.cc:85] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 4
I20260709 13:16:49.630124 11473 pending_rounds.cc:107] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.5
W20260709 13:16:49.630211 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 84051.2us
I20260709 13:16:49.547163 15636 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.631639 15593 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:49.632647 15593 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.634140 15636 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.630362 11473 raft_consensus.cc:2889] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.554296 15080 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.614018 15754 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.615254 15698 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:49.629073 15590 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.594655 15505 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.637907 15765 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:49.641299 15461 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:49.646589 15590 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.647107 15461 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.647639 15461 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:49.644925 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:49.625820 15726 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.625944 15097 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.628111 15526 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:49.655151 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.631683 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:49.627377 15658 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:49.619561 15688 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:49.605827 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.677646 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.667061 11473 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:49.705336 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:49.705569 15760 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:16:49.636757 15674 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.705955 15450 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.706501 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.707185 15765 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:49.707533 15080 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:49.667100 15756 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 13:16:49.711333 15775 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:49.711761 15452 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.652474 15703 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.712229 15505 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:49.713119 15763 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:16:49.713171 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:49.714322 15764 log.cc:927] Time spent T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:16:49.714736 15461 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.644941 15099 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:16:49.712373 15590 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:49.720009 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2378) took 1641 ms. Trace:
I20260709 13:16:49.777334 11471 rpcz_store.cc:276] 0709 13:16:48.078272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.402255 (+323983us) service_pool.cc:224] Handling call
0709 13:16:48.439799 (+ 37544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.953467 (+513668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.719694 (+766227us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b5400059954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:49.719947 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176163072}
W20260709 13:16:49.720947 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.723320 15565 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:49.730207 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.755985 15177 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:49.666445 15671 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:49.728871 15675 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.788102 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.757762 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:49.673197 15658 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:49.766103 15775 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:49.764917 15697 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:49.666640 15348 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
W20260709 13:16:49.769703 15759 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:16:49.752600 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.805687 11544 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
W20260709 13:16:49.784170 11441 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.816s (SENT)
I20260709 13:16:49.808772 11441 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:49.787195 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 212571us
I20260709 13:16:49.710479 15636 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:49.741324 15526 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:49.794538 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:49.804807 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.810189 12083 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:49.766064 15452 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.755497 15705 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.832161 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:49.844345 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.847126 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 263107us
W20260709 13:16:49.829238 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50749.2us
I20260709 13:16:49.786441 15751 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.801955 15670 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.788791 15671 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:49.789357 15704 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
I20260709 13:16:49.837855 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.872076 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.840533 15703 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.863178 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.852548 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:49.843832 15450 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:49.802182 15675 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:16:49.844787 15768 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.867329 15099 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:49.884395 11545 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:49.881557 11469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.014s	sys 0.022s
I20260709 13:16:49.865108 15705 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:49.886619 11469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.014s	sys 0.022s
I20260709 13:16:49.887133 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:49.887910 11477 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:49.865365 15473 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.868198 15668 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:49.881102 15688 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.891294 15668 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:49.891940 15766 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:16:49.893082 15704 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.893949 15668 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.883726 15099 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:49.895958 15099 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.897348 15099 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:49.899186 15473 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.901443 15473 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.901630 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:49.851780 15177 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.903805 11551 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:49.870503 15671 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:49.868018 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.922811 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:49.897384 15667 consensus_meta.cc:287] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.003s
I20260709 13:16:49.930685 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:49.923744 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.943337 15667 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.860124 15765 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:49.985971 15776 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:49.986317 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.986586 15776 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:49.979416 15474 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.639s	user 0.013s	sys 0.000s
I20260709 13:16:49.972440 15674 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:49.993981 11543 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:50.005882 15461 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:49.927006 15775 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:49.939427 15526 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:50.086490 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.087122 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59756.9us
I20260709 13:16:50.096518 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.098150 11396 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:50.100203 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:50.086800 15788 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:16:50.086499 15792 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:16:50.105682 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:50.086784 15773 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:16:50.112154 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.097904 15785 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.020877 15688 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 13:16:50.118444 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 144873us
I20260709 13:16:50.128036 15799 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.128608 12115 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:50.102596 15794 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.101055 15718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.022s	sys 0.007s
W20260709 13:16:50.145560 15718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.378s	user 0.023s	sys 0.007s
I20260709 13:16:50.109675 15452 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:50.150126 15097 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.122035 15785 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.156473 15774 consensus_queue.cc:1059] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:50.156849 15794 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.159050 15794 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.160971 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:50.164384 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 190346us
I20260709 13:16:50.165959 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:50.136955 15769 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:50.167869 11365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.159606 15774 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:16:50.175199 15695 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 2.165s	user 0.005s	sys 0.007s
I20260709 13:16:50.171073 15636 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.175791 15636 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.176632 15800 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.170343 15769 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.177489 15769 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:50.177938 15636 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.181089 15658 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.186569 15806 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:50.187109 15806 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:50.187470 15806 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:50.189456 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.196535 15636 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:50.196519 15774 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:50.198482 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 2028) took 1900 ms. Trace:
I20260709 13:16:50.198889 11400 rpcz_store.cc:276] 0709 13:16:48.298426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.298572 (+   146us) service_pool.cc:224] Handling call
0709 13:16:48.312837 (+ 14265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.384165 (+ 71328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.198257 (+1814092us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b54001e00d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:50.198424 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20564352}
I20260709 13:16:50.199887 15774 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:50.202636 15636 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:50.203639 11544 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:50.204339 15806 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.204708 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.196409 15772 consensus_queue.cc:1048] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:16:50.205677 15636 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:50.164104 15785 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.207542 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.143254 15674 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:16:50.210809 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:50.217521 11547 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:50.221345 15097 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:50.221630 15789 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.222646 15658 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:50.237967 15671 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.223310 15774 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:50.227135 15785 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:50.256047 15785 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.256633 15671 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.258289 15785 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:50.227876 15636 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:50.229605 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:50.233666 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.261375 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.262468 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.263361 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:50.263965 12123 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.234339 15327 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:50.168947 15705 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:16:50.209416 15776 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.222357 15789 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.240051 15146 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:50.247404 15683 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 13:16:50.128635 15799 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:50.248108 11298 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 244) took 31315 ms (client timeout 15000 ms). Trace:
W20260709 13:16:50.265658 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.267695 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:50.267815 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.268338 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:50.288285 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.323920 11470 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
W20260709 13:16:50.291671 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.263415 15452 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.287180 12117 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:16:50.264273 15791 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:50.327450 12117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 652) took 1156 ms. Trace:
I20260709 13:16:50.292797 12123 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:50.287968 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:50.294474 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 181631us
I20260709 13:16:50.288620 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.329815 11475 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:50.308607 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.305766 15800 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:50.312796 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:50.314215 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:50.314917 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55074 (request call id 642) took 1322 ms. Trace:
I20260709 13:16:50.315431 15739 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:50.318673 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:50.291074 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.298355 15774 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:50.293587 15327 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:50.284309 15806 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.327845 12117 rpcz_store.cc:276] 0709 13:16:49.171317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.176221 (+  4904us) service_pool.cc:224] Handling call
0709 13:16:50.286698 (+1110477us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800004598. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4f8c 00007f8da67f4ef6 0000000000375939 00007f8dae8681e0 00007f8dae9310c4 00007f8dae931131 00007f8dae8f2fc0 00007f8dae8dd929 00007f8dae8dd3d2 00007f8da7e54bd4
0709 13:16:50.286884 (+   186us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:50.327169 (+ 40285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.327397 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3033344}
I20260709 13:16:50.332732 12118 rpcz_store.cc:276] 0709 13:16:48.992508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.001502 (+  8994us) service_pool.cc:224] Handling call
0709 13:16:50.314717 (+1313215us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b540075f654. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:50.314855 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8562048}
I20260709 13:16:50.335673 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.334604 11471 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 9: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:50.334069 15791 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:50.341347 11393 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:50.322675 11298 rpcz_store.cc:269] 0709 13:16:18.932683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.947040 (+ 14357us) service_pool.cc:224] Handling call
0709 13:16:50.247890 (+31300850us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:33.622358 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:33.746474 (+124116us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:33.749434 (+  2960us) write_op.cc:435] Acquired schema lock
0709 13:16:33.752589 (+  3155us) tablet.cc:662] Decoding operations
0709 13:16:33.860681 (+108092us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:33.860738 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:33.860765 (+    27us) tablet.cc:685] Acquiring locks for 34 operations
0709 13:16:33.909664 (+ 48899us) tablet.cc:701] Row locks acquired
0709 13:16:33.909691 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:16:33.942353 (+ 32662us) write_op.cc:270] Start()
0709 13:16:34.025195 (+ 82842us) write_op.cc:276] Timestamp: P: 1783602993934703 usec, L: 0
0709 13:16:34.025224 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:16:34.360875 (+335651us) log.cc:844] Serialized 13122 byte log entry
0709 13:16:34.532016 (+171141us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da680f0ac 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b
0709 13:16:35.921281 (+1389265us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b5400018720. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da988edbe 00007f8da9899930 00007f8da98998d8 00007f8da9899858 00007f8da9899813 00007f8da9898a9c 00007f8daf784626 00007f8daf77edd8 00007f8da98d57cb 00007f8da98d82c0
0709 13:16:36.532452 (+611171us) op_driver.cc:464] REPLICATION: finished
0709 13:16:36.996947 (+464495us) write_op.cc:301] APPLY: starting
0709 13:16:36.997086 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:36.998101 (+  1015us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:36.998150 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:37.450846 (+452696us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:37.453494 (+  2648us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=34
0709 13:16:37.453800 (+   306us) write_op.cc:312] APPLY: finished
0709 13:16:37.530110 (+ 76310us) log.cc:844] Serialized 224 byte log entry
0709 13:16:37.753425 (+223315us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:37.766533 (+ 13108us) write_op.cc:454] Released schema lock
0709 13:16:37.766633 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:37.767071 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:37.767103 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":981162,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":464085,"apply.run_cpu_time_us":89596,"apply.run_wall_time_us":770123,"num_ops":34,"prepare.queue_time_us":101058,"prepare.run_cpu_time_us":47941,"prepare.run_wall_time_us":1943964,"raft.queue_time_us":596885,"raft.run_cpu_time_us":37618,"raft.run_wall_time_us":359603,"replication_time_us":2401478,"spinlock_wait_cycles":88360192,"thread_start_us":1716878,"threads_started":5,"wal-append.queue_time_us":1376877,"wal-append.run_cpu_time_us":89231,"wal-append.run_wall_time_us":2756099}]]}
I20260709 13:16:50.345589 11365 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.334692 12119 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:50.337006 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:50.348889 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.349710 11542 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:50.350862 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.353243 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:50.356092 11542 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:50.357055 11475 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:50.371794 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.360519 15518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.015s	sys 0.018s
I20260709 13:16:50.348915 15791 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.348203 15516 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:50.336993 15452 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.322971 15683 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.360647 15774 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:50.384989 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.355743 15528 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:50.387346 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.390125 15770 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.751s	user 0.001s	sys 0.001s
I20260709 13:16:50.372020 15562 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:50.405936 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.343899 15739 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:50.379660 15518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.015s	sys 0.018s
I20260709 13:16:50.366968 15671 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:16:50.412792 11397 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:50.360105 15505 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:16:50.403891 15528 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.404563 15774 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:50.419358 11400 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:50.409152 15641 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:50.418248 15528 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:50.424485 15641 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:50.424953 15641 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.407594 15802 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.428112 15799 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.428274 15704 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:50.428443 15565 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:50.428714 15704 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:50.429617 15704 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.430169 15789 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.430871 15704 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.415359 11472 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:50.407498 15562 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.440580 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:50.441435 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.441128 11401 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:16:50.440481 11394 raft_consensus.cc:1217] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Deduplicated request from leader. Original: 7.5->[8.6-8.6]   Dedup: 8.6->[]
I20260709 13:16:50.441515 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.441776 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.448311 11394 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:16:50.449923 11393 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:50.444008 11547 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:50.450346 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.446471 11395 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:50.451134 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.451436 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.451979 11402 raft_consensus.cc:2410] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:16:50.454723 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.456298 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:50.457579 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:50.441329 15755 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.154s	user 0.004s	sys 0.000s
I20260709 13:16:50.461534 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.459602 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.462750 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.465327 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.458863 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.466158 11401 raft_consensus.cc:2410] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:50.446018 15774 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:50.407402 15807 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:16:50.445940 15562 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:50.471635 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.440874 15683 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:50.460408 15765 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:50.440173 15739 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:50.480365 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.475888 11393 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:50.473206 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 563) took 1120 ms. Trace:
I20260709 13:16:50.482321 11550 rpcz_store.cc:276] 0709 13:16:49.353109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.358062 (+  4953us) service_pool.cc:224] Handling call
0709 13:16:49.371988 (+ 13926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.372360 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:49.372390 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.372823 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.584397 (+211574us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:49.584436 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.584504 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.389537 (+805033us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:50.467895 (+ 78358us) raft_consensus.cc:1685] finished
0709 13:16:50.467915 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.473110 (+  5195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":11299,"spinlock_wait_cycles":4745088,"thread_start_us":154877,"threads_started":1,"wal-append.queue_time_us":237344}
W20260709 13:16:50.486099 15810 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:16:50.434963 15769 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:50.478947 11442 connection.cc:256] RPC call timeout handler was delayed by -1.3239702349999334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:16:50.492854 11442 connection.cc:256] RPC call timeout handler was delayed by -1.3239702349999334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:16:50.493338 11442 connection.cc:256] RPC call timeout handler was delayed by -1.2051702349999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:16:50.493587 11442 connection.cc:256] RPC call timeout handler was delayed by -1.1133702350000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 13:16:50.494601 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.495708 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.490634 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.491636 11401 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:50.491533 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.501132 11395 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:50.476586 15668 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:50.481428 15785 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:50.510291 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.510206 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.481617 15674 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.511533 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.511554 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.513130 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.513440 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.516202 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.513338 11395 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:50.512110 11471 raft_consensus.cc:2376] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9. Re-sending same reply.
I20260709 13:16:50.518857 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.521878 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.522418 11398 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:50.522620 11472 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:50.519474 11471 raft_consensus.cc:2468] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:50.523578 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.523718 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.524428 11550 raft_consensus.cc:2410] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:50.524942 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:50.524812 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.525728 11471 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:50.440755 15327 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.529877 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.531919 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.532801 11401 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:50.550724 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.551203 11550 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:50.552343 11393 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:50.560719 15452 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:50.515793 15674 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.569399 15674 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.515980 15668 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.572602 11396 raft_consensus.cc:3055] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:50.573179 11396 raft_consensus.cc:740] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:50.573056 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:50.552174 15813 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.003s
I20260709 13:16:50.636622 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.594885 11396 consensus_queue.cc:260] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:50.527197 15785 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:50.638885 11288 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.331s	user 0.010s	sys 0.008s
I20260709 13:16:50.637415 11393 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:50.557060 15327 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:50.646248 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:50.573879 15808 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:16:50.654176 15769 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.659452 11542 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:16:50.659621 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.659857 11542 pending_rounds.cc:85] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 5
I20260709 13:16:50.660147 11542 pending_rounds.cc:107] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:16:50.660382 11542 raft_consensus.cc:2889] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.662222 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.664201 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.665575 11469 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:50.666396 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.666980 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:50.667539 11396 raft_consensus.cc:3060] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:50.667769 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.670807 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:50.671665 11474 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:50.646323 15756 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:16:50.675556 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:50.678149 11401 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:50.679535 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:50.686152 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.686405 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.697101 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:50.701201 11474 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:50.692296 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:50.679788 15674 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:50.704082 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:50.707234 11476 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:50.709594 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:50.711331 15819 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 13:16:50.713078 15814 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:16:50.717396 15762 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.081s	user 0.000s	sys 0.002s
I20260709 13:16:50.718606 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2391) took 1413 ms. Trace:
I20260709 13:16:50.719080 11548 rpcz_store.cc:276] 0709 13:16:49.304878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.305035 (+   157us) service_pool.cc:224] Handling call
0709 13:16:49.305335 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.305578 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:49.305617 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.307622 (+  2005us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.422170 (+114548us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:49.422236 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.422327 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.716825 (+1294498us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:50.718225 (+  1400us) raft_consensus.cc:1685] finished
0709 13:16:50.718256 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.718522 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8264576,"thread_start_us":114130,"threads_started":1,"wal-append.queue_time_us":275558}
I20260709 13:16:50.726164 15634 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:50.726636 15634 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:50.727592 15634 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.740290 15785 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.740939 15785 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:50.743477 15668 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:50.705538 15779 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:16:50.745620 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.710425 11474 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:16:50.746184 11474 pending_rounds.cc:85] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:16:50.746469 11474 pending_rounds.cc:107] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:16:50.746793 11474 raft_consensus.cc:2889] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.753432 15634 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:50.696949 15538 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.696748 15452 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:50.738725 15641 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.770543 15452 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.710278 15805 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:50.703153 15658 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:16:50.943264 15768 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:51.009917 15817 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
W20260709 13:16:50.955049 15528 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.002s
I20260709 13:16:50.788825 15538 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:50.953881 15692 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.000142 15768 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:51.000453 15562 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:51.127552 15538 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:51.083159 15528 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:51.096339 15692 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:51.355629 15774 consensus_meta.cc:287] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.874s	user 0.010s	sys 0.000s
I20260709 13:16:51.299758 15452 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:51.343685 15634 consensus_meta.cc:287] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
W20260709 13:16:51.382691 15791 consensus_meta.cc:287] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.991s	user 0.005s	sys 0.005s
I20260709 13:16:51.356310 15774 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:51.482249 15562 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.423275 15791 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:51.514034 15528 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.515255 15791 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.512921 15692 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.514892 15774 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.524425 15528 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:51.527931 15528 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:51.530213 15528 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.531400 15768 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:51.532135 15641 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:51.540532 15327 consensus_meta.cc:287] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.880s	user 0.008s	sys 0.004s
W20260709 13:16:51.546351 15765 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.066s	user 0.008s	sys 0.000s
I20260709 13:16:51.546847 15765 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:51.676144 15825 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:16:51.849644 15768 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:51.850473 15768 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:51.934718 15768 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:52.412412 11477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.012s	sys 0.006s
W20260709 13:16:52.413074 11477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.013s	sys 0.006s
W20260709 13:16:52.449012 12083 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.040s	sys 0.050s
W20260709 13:16:52.449712 12083 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.040s	sys 0.050s
W20260709 13:16:52.450105 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 953880us
W20260709 13:16:52.643868 12083 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.923s (SENT) [suppressed 20 similar messages]
W20260709 13:16:52.741302 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97482.8us
I20260709 13:16:52.748206 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2415) took 2481 ms. Trace:
I20260709 13:16:52.748734 11477 rpcz_store.cc:276] 0709 13:16:50.267139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.268516 (+  1377us) service_pool.cc:224] Handling call
0709 13:16:50.303844 (+ 35328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.304261 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:50.304300 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:50.304892 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:16:51.539741 (+1234849us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180049b3a0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:52.413630 (+873889us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:52.574203 (+160573us) log.cc:844] Serialized 14 byte log entry
0709 13:16:52.747734 (+173531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.747853 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:52.747897 (+    44us) raft_consensus.cc:1685] finished
0709 13:16:52.747988 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.748130 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":160513,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":164491,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":202,"spinlock_wait_cycles":3237760,"thread_start_us":1206604,"threads_started":3,"wal-append.queue_time_us":2104653}
I20260709 13:16:52.752476 15837 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:52.753116 15837 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.756327 15837 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:52.757015 15837 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:52.757753 15837 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:52.760489 15837 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:52.760970 11544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.184s	user 0.022s	sys 0.045s
W20260709 13:16:52.769982 11544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.198s	user 0.024s	sys 0.046s
W20260709 13:16:52.781158 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.056s	user 0.007s	sys 0.022s
I20260709 13:16:52.784343 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1672) took 2109 ms. Trace:
W20260709 13:16:52.782749 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.058s	user 0.009s	sys 0.024s
I20260709 13:16:52.785068 11474 rpcz_store.cc:276] 0709 13:16:50.675056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.747754 (+ 72698us) service_pool.cc:224] Handling call
0709 13:16:52.783583 (+2035829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.783835 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.783876 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.784000 (+   124us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:52.784047 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.784118 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.784263 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:52.795251 15080 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:52.795841 15080 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.797380 15080 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:52.797717 15837 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.798393 15080 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:52.798286 15837 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.799124 15080 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:52.800843 15837 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:52.803298 11474 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:16:52.803635 15840 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:52.804155 15840 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:52.804525 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:52.805137 11474 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:16:52.806273 15840 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:52.808135 15080 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:52.809069 15688 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:52.809564 15688 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:52.810467 15800 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:52.810968 15800 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.811610 15688 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:52.812155 15688 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.812554 15688 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.813104 15800 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:52.813148 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2425) took 2624 ms. Trace:
I20260709 13:16:52.814464 15688 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:52.815357 15688 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:52.816051 15839 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:52.815917 15688 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:52.822723 15688 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:52.813750 11544 rpcz_store.cc:276] 0709 13:16:50.188407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.205575 (+ 17168us) service_pool.cc:224] Handling call
0709 13:16:50.205842 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.206034 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:50.220007 (+ 13973us) log.cc:844] Serialized 14 byte log entry
0709 13:16:50.571658 (+351651us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b1800462ac0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b923a 00007f8da9923a2a
0709 13:16:52.793303 (+2221645us) log.cc:844] Serialized 14 byte log entry
0709 13:16:52.812804 (+ 19501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.812859 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:52.812907 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.812974 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.813091 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43992,"prepare.queue_time_us":20110,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":82,"raft.queue_time_us":19837,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"spinlock_wait_cycles":112178560,"thread_start_us":2239842,"threads_started":4,"wal-append.queue_time_us":1357784,"wal-append.run_cpu_time_us":2081,"wal-append.run_wall_time_us":1118755}
I20260709 13:16:52.826865 15099 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:52.827359 15099 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:52.828220 15099 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:52.829493 15099 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:52.833067 11366 connection.cc:256] RPC call timeout handler was delayed by -2.0763528929999211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:16:52.834156 11366 connection.cc:256] RPC call timeout handler was delayed by -1.6561187730001166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:16:52.876808 15841 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.877532 15841 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.171638 15841 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.172447 15841 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.173043 15841 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:53.307076 11400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50180 (request call id 608) took 2971 ms (client timeout 1970 ms). Trace:
W20260709 13:16:53.439250 11400 rpcz_store.cc:269] 0709 13:16:50.335279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.335398 (+   119us) service_pool.cc:224] Handling call
0709 13:16:53.306112 (+2970714us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b5400230854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:53.306268 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170260224}
W20260709 13:16:53.434868 11516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.732s	user 0.043s	sys 0.054s
W20260709 13:16:53.434772 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 500036us
I20260709 13:16:53.440724 15841 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.441287 15841 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.441743 15841 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:53.437511 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.443061 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.443523 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.444043 11441 connection.cc:256] RPC call timeout handler was delayed by -1.3864060020000579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 13:16:53.444984 11516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.742s	user 0.044s	sys 0.054s
I20260709 13:16:53.444881 15841 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:53.445466 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 2.75306e+06us
I20260709 13:16:53.447969 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:53.458706 15129 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:53.461958 11469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.756s	user 0.007s	sys 0.029s
W20260709 13:16:53.462564 11469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.756s	user 0.008s	sys 0.030s
I20260709 13:16:53.467933 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1649) took 2973 ms. Trace:
I20260709 13:16:53.469275 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.468293 11469 rpcz_store.cc:276] 0709 13:16:50.494852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.705017 (+210165us) service_pool.cc:224] Handling call
0709 13:16:50.705388 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.705625 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:50.705905 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 13:16:53.467578 (+2761673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:53.467633 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:53.467684 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.467749 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.467872 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2638,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4857,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":2761255,"threads_started":2,"wal-append.queue_time_us":821264,"wal-append.run_cpu_time_us":1156,"wal-append.run_wall_time_us":1001172}
I20260709 13:16:53.470773 15852 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:53.471397 15852 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.471776 15852 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:53.473500 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.474315 15129 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:53.474752 15129 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.474617 15593 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:53.475183 15129 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:53.475505 15593 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:53.475780 15691 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:53.475860 15593 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:53.476311 15691 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:53.471946 15851 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:16:53.478232 11400 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:53.479581 11400 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:16:53.479998 15851 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:53.480777 11400 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:53.481129 15691 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.482012 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:53.482673 11401 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:53.482596 15851 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:53.483269 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:53.483779 15851 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:53.483834 11398 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:53.484300 15851 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:53.485137 15851 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.486073 15851 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:53.486483 11399 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:53.476851 15593 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.488256 15593 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.494228 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.495179 15853 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:53.495648 15837 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:53.496351 15837 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.498502 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.509471 15853 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:53.509667 15860 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:53.509954 15853 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:53.510286 15860 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.510787 15860 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.510885 15853 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.512372 15853 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:53.517809 15860 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.508668 15851 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:53.519445 15851 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:53.519809 15851 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.529781 15851 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.533339 15129 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.535916 15851 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:53.536365 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 655) took 3273 ms. Trace:
I20260709 13:16:53.536779 12119 rpcz_store.cc:276] 0709 13:16:50.263349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.289070 (+ 25721us) service_pool.cc:224] Handling call
0709 13:16:50.330634 (+ 41564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.346321 (+ 15687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.536150 (+3189829us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54006c0f50. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:53.536273 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94770432}
I20260709 13:16:53.535646 15861 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:16:53.540098 15593 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:53.539819 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:53.541929 15593 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:53.542333 11398 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.542881 15593 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.544070 15593 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:53.546769 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.555299 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.560495 11365 connection.cc:256] RPC call timeout handler was delayed by -1.5795468630001324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:16:53.561147 11365 connection.cc:256] RPC call timeout handler was delayed by -1.3856979769998361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 13:16:53.561712 11365 connection.cc:256] RPC call timeout handler was delayed by -1.2992979769999238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 13:16:53.562186 11365 connection.cc:256] RPC call timeout handler was delayed by -1.0504661950001264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2315s for a response
I20260709 13:16:53.564741 11398 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:53.566042 15860 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:53.566766 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.567298 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:53.572937 11398 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.566711 15849 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:53.573640 15849 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.575850 15849 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.576455 15849 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.576978 15849 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.579636 15129 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:53.580536 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.580879 15800 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.581561 15800 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.580542 15841 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.642037 15844 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.641829 15841 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.644855 15841 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.582530 15692 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:53.581424 15666 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.645757 15692 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.649293 15692 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.649923 15692 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.642606 15844 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.650460 15692 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.652901 15844 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.627028 15849 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:53.640740 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.674798 15851 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:53.675494 15851 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:53.676522 12085 connection.cc:256] RPC call timeout handler was delayed by -1.477453240999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 13:16:53.661969 11398 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:53.646124 15666 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.676038 15869 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.678354 15851 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.572093 15860 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:53.657970 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:53.681674 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:53.693396 11399 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:53.695425 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.696218 12116 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:53.697950 12116 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:53.653815 15692 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.699893 12116 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:53.696775 15869 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.702924 15869 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.693324 15852 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.704648 12116 raft_consensus.cc:1240] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:53.705235 15852 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:53.706287 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:53.706560 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.707129 12119 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 13:16:53.699352 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 58905.7us
I20260709 13:16:53.708367 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:53.709259 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.026s	user 0.022s	sys 0.065s
W20260709 13:16:53.709678 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.029s	user 0.023s	sys 0.066s
I20260709 13:16:53.710038 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:53.710213 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 3.03049e+06us
I20260709 13:16:53.711401 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.712302 15666 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:53.712751 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.713299 12115 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:53.699585 15860 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:53.714408 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.714849 15860 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.710840 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.717286 15856 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.717774 15856 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.718317 15800 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.718837 15692 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:53.719269 15692 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:53.719650 15692 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:53.719918 15856 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.720423 15856 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.720824 15856 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.723778 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:53.724869 15856 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.725327 15841 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:53.725975 15841 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:53.727253 11470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.012s	user 0.010s	sys 0.021s
I20260709 13:16:53.727056 15841 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:53.728422 11470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.013s	user 0.010s	sys 0.021s
I20260709 13:16:53.729218 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1662) took 3211 ms. Trace:
I20260709 13:16:53.729537 11470 rpcz_store.cc:276] 0709 13:16:50.518036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.713957 (+195921us) service_pool.cc:224] Handling call
0709 13:16:50.714233 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.714426 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:50.714891 (+   465us) log.cc:844] Serialized 14 byte log entry
0709 13:16:53.728902 (+3014011us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:53.728954 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:53.728994 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.729055 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.729167 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"thread_start_us":3013221,"threads_started":1,"wal-append.queue_time_us":962457,"wal-append.run_cpu_time_us":1625,"wal-append.run_wall_time_us":1020591}
I20260709 13:16:53.731753 11399 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:53.707119 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:53.734421 12116 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:53.749776 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.751757 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:53.753597 15099 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.010s
I20260709 13:16:53.761276 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.764401 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:53.787529 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.788131 11544 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:53.788931 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:53.789855 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.765033 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.790756 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.791754 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.791872 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.792346 11543 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:53.793185 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.793893 11543 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:53.765715 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:53.766369 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:53.762699 15692 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.794945 15837 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.797430 12119 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:53.800675 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.801733 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.802261 12119 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:16:53.798683 12118 raft_consensus.cc:2393] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:53.796312 11550 raft_consensus.cc:2410] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:16:53.804283 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:53.805588 11550 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:53.797653 12115 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:53.811317 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.811949 12119 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
W20260709 13:16:53.812770 15804 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 3.396s	user 0.020s	sys 0.016s
I20260709 13:16:53.813062 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.737493 15856 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:53.814162 15856 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:53.814579 15856 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:53.794631 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:53.829682 11471 consensus_meta.cc:287] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 3.303s	user 0.009s	sys 0.015s
I20260709 13:16:53.830185 11471 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
W20260709 13:16:53.830778 11471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1651) took 3335 ms (client timeout 1925 ms). Trace:
W20260709 13:16:53.831233 11471 rpcz_store.cc:269] 0709 13:16:50.495402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.524369 (+ 28967us) service_pool.cc:224] Handling call
0709 13:16:53.830679 (+3306310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.833061 11547 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:53.765443 11288 maintenance_manager.cc:643] P d085878dd61a4a3984dff72a53c92b4d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.133s	user 0.216s	sys 0.103s Metrics: {"bytes_written":29389,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":367313,"dirs.run_cpu_time_us":3280,"dirs.run_wall_time_us":1164917,"drs_written":1,"lbm_read_time_us":487,"lbm_reads_lt_1ms":4,"lbm_write_time_us":178543,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"spinlock_wait_cycles":84290688,"thread_start_us":400450,"threads_started":2,"update_count":3,"wal-append.queue_time_us":210642,"wal-append.run_cpu_time_us":748,"wal-append.run_wall_time_us":1112569}
I20260709 13:16:53.772878 15852 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:16:53.834172 15852 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.773144 15593 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:53.773337 15851 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:53.835254 15593 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.835345 15851 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.837182 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.837652 15851 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.838107 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.838191 15593 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:53.838769 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6140 after lost signal to thread 10255
W20260709 13:16:53.779142 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.840193 11367 connection.cc:256] RPC call timeout handler was delayed by -1.6067297639999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 13:16:53.846855 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 136075us
I20260709 13:16:53.847582 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.849491 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:53.838979 15593 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:53.856848 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.840678 15852 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.847194 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:53.858434 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:53.858608 15099 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:53.859246 15875 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:53.859965 15837 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:53.860265 15875 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:53.860890 15875 consensus_queue.cc:1243] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:53.826664 11542 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:53.859392 15099 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:53.859258 11289 maintenance_manager.cc:419] P d085878dd61a4a3984dff72a53c92b4d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8478 bytes on disk
I20260709 13:16:53.862416 15099 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.839181 15851 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:53.864040 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:53.864281 11544 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:53.864411 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:53.864653 11400 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:16:53.861387 15837 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:53.866755 15837 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:53.867982 15837 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.869805 15837 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:53.799588 15692 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.872426 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.876149 15875 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:53.861758 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:53.876724 15875 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.878844 15874 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:53.879122 15875 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.882448 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.883237 11474 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:53.885051 11474 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:53.886534 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.887200 11474 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:53.889077 11288 maintenance_manager.cc:643] P d085878dd61a4a3984dff72a53c92b4d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":415,"lbm_reads_lt_1ms":4}
I20260709 13:16:53.891104 11474 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:53.891436 12116 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:16:53.892266 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.891652 11474 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:53.893896 11474 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.895316 11474 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:53.897743 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:53.898342 11470 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:53.899917 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:53.900578 11469 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:53.896522 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.857157 15593 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:53.903432 11514 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 3.225s	user 0.041s	sys 0.032s
W20260709 13:16:53.903839 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.903878 11514 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 3.226s	user 0.041s	sys 0.032s
W20260709 13:16:53.904789 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 3.22673e+06us
I20260709 13:16:53.904735 15851 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:53.905311 12114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 788) took 4650 ms (client timeout 1613 ms). Trace:
W20260709 13:16:53.905722 12114 rpcz_store.cc:269] 0709 13:16:49.255391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.259546 (+  4155us) service_pool.cc:224] Handling call
0709 13:16:53.905101 (+4645555us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54001d0bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:53.905218 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619880704}
I20260709 13:16:53.906286 15851 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.907488 15080 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.908015 15080 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.910272 15080 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.913123 15697 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.913614 15697 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.903285 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:53.917168 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.902738 15593 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.919339 11468 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:53.936995 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.937827 11477 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:53.910831 15080 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:53.938750 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.938728 11474 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:16:53.940713 11470 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
W20260709 13:16:53.942626 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.943799 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.946201 11474 pending_rounds.cc:85] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:16:53.947475 11474 pending_rounds.cc:107] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 9.7
W20260709 13:16:53.946980 11514 connection.cc:256] RPC call timeout handler was delayed by -3.1146064800000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
I20260709 13:16:53.947849 11474 raft_consensus.cc:2889] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:53.948580 11514 connection.cc:256] RPC call timeout handler was delayed by -3.0741064799999549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
I20260709 13:16:53.904990 15856 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:53.947067 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.949662 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.950237 11477 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 6.
I20260709 13:16:53.949503 15718 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.956298 15718 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.958452 15856 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:53.959671 15718 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:53.959856 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.960477 12114 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:16:53.963908 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:53.964542 11400 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:53.965128 11469 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:16:53.966023 11514 connection.cc:256] RPC call timeout handler was delayed by -3.0237064800001008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:16:53.967442 11514 connection.cc:256] RPC call timeout handler was delayed by -2.99850648000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:16:53.967758 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.968377 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 90795.6us
W20260709 13:16:53.968717 11514 connection.cc:256] RPC call timeout handler was delayed by -2.94900647999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20640000000000003s for a response
I20260709 13:16:53.968811 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.969738 11469 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:53.971056 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.971652 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:53.972571 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.969738 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:53.971647 11469 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:53.967756 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:53.977442 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.979271 15691 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:53.979578 15751 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.980245 15852 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:53.980146 15751 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:53.980671 15852 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.981065 15852 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:53.983894 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.984148 15751 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:53.984707 11514 connection.cc:256] RPC call timeout handler was delayed by -2.7969064799999614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260709 13:16:53.985049 11514 connection.cc:256] RPC call timeout handler was delayed by -2.7812062299999525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 13:16:53.985332 11514 connection.cc:256] RPC call timeout handler was delayed by -2.770406229999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:16:53.985595 11514 connection.cc:256] RPC call timeout handler was delayed by -2.3311979869999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 13:16:53.985863 11514 connection.cc:256] RPC call timeout handler was delayed by -2.159706480000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2941s for a response
I20260709 13:16:53.985987 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:53.991591 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:53.992291 11470 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:53.949815 15851 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:53.995107 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:54.003096 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.003801 11476 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:16:54.004480 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.004652 11476 pending_rounds.cc:85] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 5
I20260709 13:16:54.003432 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.005174 11476 pending_rounds.cc:107] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:16:54.005335 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.005652 15882 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:54.005183 15466 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:54.005645 11476 raft_consensus.cc:2889] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:54.007457 11469 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:54.007635 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1653) took 3511 ms. Trace:
I20260709 13:16:53.975715 11546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2432) took 3711 ms. Trace:
I20260709 13:16:54.007334 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:54.008421 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.008924 12118 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:54.009342 11477 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:54.009270 11469 raft_consensus.cc:2410] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 14 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
W20260709 13:16:54.009795 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 113352us
I20260709 13:16:53.997160 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.008199 11546 rpcz_store.cc:276] 0709 13:16:50.264367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.291600 (+ 27233us) service_pool.cc:224] Handling call
0709 13:16:50.291906 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.411075 (+119169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:50.411115 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:50.411725 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:16:50.512685 (+100960us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:50.512724 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.458352 (+2945628us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b480002d9d8. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:53.975447 (+517095us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54002c65d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:53.975501 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:53.975543 (+    42us) raft_consensus.cc:1685] finished
0709 13:16:53.975558 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.975675 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266,"raft.queue_time_us":1195,"raft.run_cpu_time_us":30993,"raft.run_wall_time_us":3045964,"spinlock_wait_cycles":276573824,"thread_start_us":88529,"threads_started":1,"wal-append.queue_time_us":231062,"wal-append.run_cpu_time_us":1617,"wal-append.run_wall_time_us":1016507}
I20260709 13:16:54.008118 11476 rpcz_store.cc:276] 0709 13:16:50.495918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.751399 (+255481us) service_pool.cc:224] Handling call
0709 13:16:53.999295 (+3247896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.007401 (+  8106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.007572 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:54.011508 11400 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:53.996662 11402 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:53.986194 11514 connection.cc:256] RPC call timeout handler was delayed by -2.1219064800000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2983s for a response
W20260709 13:16:54.012929 11514 connection.cc:256] RPC call timeout handler was delayed by -1.9674461529998553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:16:54.013273 11514 connection.cc:256] RPC call timeout handler was delayed by -1.9644166989999121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 13:16:54.013548 11514 connection.cc:256] RPC call timeout handler was delayed by -1.879406230000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.262s for a response
W20260709 13:16:54.013827 11514 connection.cc:256] RPC call timeout handler was delayed by -1.645141237999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 13:16:54.014328 11514 connection.cc:256] RPC call timeout handler was delayed by -1.5318635470000572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.252s for a response
I20260709 13:16:53.946192 15080 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:54.007108 11474 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:54.022400 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:54.023947 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.025923 15849 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:54.026909 15852 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.028791 11477 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:54.031807 11474 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:54.032184 11474 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:53.979316 15718 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:54.019876 15688 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:53.976222 15837 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:54.031534 15466 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:54.032850 11474 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.049822 11474 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:53.995179 15875 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:16:54.025120 15800 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.063256 15800 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.063750 15800 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.047878 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.064162 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.064647 11476 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:16:54.054659 11468 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:53.994927 15851 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:54.019726 15751 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:16:54.067993 11514 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.618s (SENT)
I20260709 13:16:54.092849 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:54.069555 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.050449 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.094444 11477 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
W20260709 13:16:54.338480 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.095952 11514 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.630s (SENT)
I20260709 13:16:54.341727 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:54.342808 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 246869us
W20260709 13:16:54.346586 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.347783 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08000beb80 after lost signal to thread 11334
I20260709 13:16:54.048401 15688 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.160436 15851 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:54.349380 15688 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.103744 15861 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:54.350437 15861 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:54.314385 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 205020us
I20260709 13:16:54.162685 15667 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:54.257774 15800 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:54.354259 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 322724us
I20260709 13:16:54.351408 15861 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:54.126233 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.096829 15129 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:54.048593 15718 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.163568 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 645) took 4995 ms. Trace:
I20260709 13:16:54.230355 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 562) took 4877 ms. Trace:
I20260709 13:16:54.360069 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:54.361529 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 311413us
I20260709 13:16:54.363559 12112 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:54.365043 11545 rpcz_store.cc:276] 0709 13:16:49.352782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.764005 (+411223us) service_pool.cc:224] Handling call
0709 13:16:49.764675 (+   670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.884953 (+120278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.229990 (+4345037us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400134cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:54.230242 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274747136}
I20260709 13:16:54.371387 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:54.372016 11400 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:54.373004 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 349105us
I20260709 13:16:54.373678 11402 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:54.376356 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.362839 12113 rpcz_store.cc:276] 0709 13:16:49.168178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.169534 (+  1356us) service_pool.cc:224] Handling call
0709 13:16:49.182198 (+ 12664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.183756 (+  1558us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:49.238081 (+ 54325us) log.cc:844] Serialized 14 byte log entry
0709 13:16:49.319922 (+ 81841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:49.320008 (+    86us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:49.320072 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.163174 (+4843102us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5400605c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:54.163260 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.163445 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48852,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":3410,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":3085696,"thread_start_us":104249,"threads_started":2,"wal-append.queue_time_us":184537,"wal-append.run_cpu_time_us":2654,"wal-append.run_wall_time_us":1513174}
I20260709 13:16:54.356866 15593 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:54.381757 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.383900 11402 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:54.384076 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.385279 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.388077 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.390296 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:54.391067 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:54.391647 11548 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:54.392373 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.393558 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.391042 11545 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.381421 15861 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.410723 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.395977 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.412922 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.405836 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:54.408248 15080 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:54.408278 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:54.447364 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:54.455122 11542 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.448642 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:54.414019 11548 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:54.456562 11469 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.394785 15336 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:54.453038 11400 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:16:54.388412 15099 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 13:16:54.428174 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.398833 15416 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:54.448148 11474 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:54.457595 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:54.543277 11547 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:54.466634 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69644.5us
W20260709 13:16:54.470764 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 57946.6us
I20260709 13:16:54.550202 11442 leader_election.cc:400] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:54.471386 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.473472 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:54.553283 12123 raft_consensus.cc:2376] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11. Re-sending same reply.
I20260709 13:16:54.553789 11547 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:54.473982 11476 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:54.555485 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.548568 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:54.457235 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 95277.6us
I20260709 13:16:54.480688 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:54.481020 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:54.561825 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.481587 11542 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:54.563835 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.564499 11542 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:54.565280 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.567313 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.561122 12113 raft_consensus.cc:2393] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:54.483647 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.496798 11469 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
W20260709 13:16:54.494565 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.571333 11469 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:54.519459 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:54.576809 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.538452 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.577546 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.578441 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.569948 11543 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:54.413184 15177 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.560170 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.584023 11470 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:54.566404 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.586475 11542 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:16:54.486512 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:54.587687 11550 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:54.523298 11300 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 7 to 8, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.571528 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.455538 15874 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:54.581089 11393 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:54.457583 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1664) took 3928 ms. Trace:
I20260709 13:16:54.558223 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:54.542201 11545 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:16:54.465339 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.539886 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:54.591796 11473 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
W20260709 13:16:54.582118 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.593297 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.590327 11475 rpcz_store.cc:276] 0709 13:16:50.529039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.724332 (+195293us) service_pool.cc:224] Handling call
0709 13:16:50.724688 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.786353 (+2061665us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:52.786402 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.185851 (+1399449us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:54.186392 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.366783 (+180391us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:54.366836 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.366967 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.447359 (+ 80392us) raft_consensus.cc:1685] finished
0709 13:16:54.447378 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.457518 (+ 10140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16550,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":3214425856,"thread_start_us":2241074,"threads_started":2,"wal-append.queue_time_us":247044}
I20260709 13:16:54.472616 15837 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:54.602773 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:54.605327 12112 raft_consensus.cc:2376] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13. Re-sending same reply.
I20260709 13:16:54.611426 11515 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.537379 15416 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.502504 15697 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:54.581405 15840 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:54.589450 15080 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:54.589013 15703 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:54.589411 15692 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:54.575626 15336 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:54.616809 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.617523 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.608021 15841 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.558821 15769 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:16:54.626312 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 176354us
I20260709 13:16:54.509193 15688 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:54.617377 15416 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:54.680230 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 208912us
I20260709 13:16:54.678785 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:54.737879 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 186544us
I20260709 13:16:54.744621 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.748127 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.740059 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.751674 11393 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:16:54.480518 15886 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.165s
I20260709 13:16:54.616596 15837 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:54.740487 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.614349 15593 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:54.743019 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.622352 15466 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:54.619064 15860 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:54.641338 15336 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:54.676460 15692 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:54.617538 15851 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.174s
I20260709 13:16:54.620311 15080 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:54.763056 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.681243 15701 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:54.796623 12112 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:54.742278 15703 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:54.809068 15885 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:54.795141 15886 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:54.811679 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.753646 15837 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:54.817917 11427 heartbeater.cc:646] Failed to heartbeat to 127.10.3.62:34815 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.10.3.62:34815 timed out after 15.000s (SENT)
I20260709 13:16:54.848488 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:54.818655 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2434) took 4553 ms. Trace:
I20260709 13:16:54.849267 11551 rpcz_store.cc:276] 0709 13:16:50.264940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.292056 (+ 27116us) service_pool.cc:224] Handling call
0709 13:16:50.297750 (+  5694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.298022 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:50.298059 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:50.298590 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:16:50.505540 (+206950us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:50.505576 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.505638 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.818415 (+4312777us) raft_consensus.cc:1685] finished
0709 13:16:54.818437 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.818583 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":152341,"threads_started":1,"wal-append.queue_time_us":219456,"wal-append.run_cpu_time_us":2208,"wal-append.run_wall_time_us":1400286}
I20260709 13:16:54.821159 15800 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:54.827960 15688 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:54.835793 11396 consensus_meta.cc:287] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 4.168s	user 0.011s	sys 0.025s
I20260709 13:16:54.618911 15892 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.145s
I20260709 13:16:54.840171 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.755190 15904 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:54.623176 15840 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.860466 11396 raft_consensus.cc:1275] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:16:54.836351 15593 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:54.879221 15593 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:54.843706 15080 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.879727 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 607) took 4770 ms. Trace:
I20260709 13:16:54.880136 11396 rpcz_store.cc:276] 0709 13:16:50.101460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.101560 (+   100us) service_pool.cc:224] Handling call
0709 13:16:50.101881 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.572469 (+470588us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b5400540c20. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9890dc0 00007f8da991b1c0 00007f8da991c231 00007f8da9918d8f 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c
0709 13:16:54.871637 (+4299168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.871842 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17339776}
I20260709 13:16:54.880578 15768 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:16:54.881196 15630 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:16:54.882216 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:54.872046 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.873674 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:54.873698 15769 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:54.875010 15703 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.875996 15892 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:54.876562 15904 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:54.869207 15701 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:54.845074 15885 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:54.851387 15837 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.847862 15886 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:54.885805 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:54.874899 15906 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:54.914740 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:54.897902 15890 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.544s	user 0.001s	sys 0.004s
I20260709 13:16:54.917666 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2453) took 1147 ms. Trace:
I20260709 13:16:54.918084 15691 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:54.920219 15905 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:54.920634 15905 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:54.902614 15688 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:54.921705 15688 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:54.922596 15688 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.898077 15692 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.926831 15754 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:54.927320 15754 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.878471 15416 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.927661 15754 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:54.929306 15416 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:54.929829 15902 log.cc:927] Time spent T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:16:54.870070 15336 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:54.931233 15336 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.931699 15701 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:54.933899 15901 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.346s	user 0.002s	sys 0.000s
I20260709 13:16:54.897493 15768 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:54.935222 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43640 (request call id 1665) took 4403 ms. Trace:
I20260709 13:16:54.935575 11395 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:16:54.942907 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.918097 11544 rpcz_store.cc:276] 0709 13:16:53.770600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.791661 (+ 21061us) service_pool.cc:224] Handling call
0709 13:16:54.917452 (+1125791us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b540041d7d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:54.917582 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165290880}
I20260709 13:16:54.939262 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.901716 15915 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:54.945997 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:54.968353 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.969515 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:54.970815 15765 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:54.971724 15765 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:54.972126 15765 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:54.972512 15765 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.942225 11472 rpcz_store.cc:276] 0709 13:16:50.531787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.741806 (+2210019us) service_pool.cc:224] Handling call
0709 13:16:52.742311 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.742728 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.742767 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.498151 (+1755384us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:54.498598 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.591156 (+ 92558us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:54.614904 (+ 23748us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.620970 (+  6066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.621070 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.933240 (+312170us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:54.935018 (+  1778us) raft_consensus.cc:1685] finished
0709 13:16:54.935038 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.935168 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5651,"prepare.queue_time_us":24141,"prepare.run_wall_time_us":2342,"raft.queue_time_us":5903,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":17664,"spinlock_wait_cycles":3331456128,"thread_start_us":92066,"threads_started":1,"wal-append.queue_time_us":88819}
I20260709 13:16:54.979735 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:54.980971 12112 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.916885 15886 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:54.982810 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 136180us
I20260709 13:16:54.976919 15769 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:54.983997 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.984957 11396 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:54.986531 11402 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:54.976492 15691 raft_consensus.cc:2764] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:16:54.991154 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 243108us
W20260709 13:16:54.988544 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 309074us
I20260709 13:16:54.990782 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:54.987174 11365 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:55.013669 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.013461 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:55.021755 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:54.991482 15416 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.981127 15768 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:55.014657 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:54.992879 11396 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:55.036288 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.036947 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.037288 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.030471 15895 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:16:55.062832 15874 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:55.062963 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.040365 15909 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.065110 15909 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.040637 15852 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:54.989241 15885 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.042210 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:55.067747 15885 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:55.067948 12113 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.038746 15861 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:16:55.043291 15912 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:55.071442 15912 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:55.072415 15912 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.075430 15909 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.075716 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.076336 11393 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:55.043238 15918 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:16:55.047225 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08003664a0 after lost signal to thread 11426
I20260709 13:16:55.024430 15910 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:55.079252 15910 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:55.079677 15910 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.016036 15913 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:55.082473 15886 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.083316 15913 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:55.084177 15913 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.085323 15765 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.054632 15840 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.087374 15765 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.087711 15718 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.035176 15080 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.089536 15080 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.090112 12113 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
W20260709 13:16:55.095372 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 125952us
W20260709 13:16:55.095958 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3ea0 after lost signal to thread 11501
W20260709 13:16:55.096280 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08003713c0 after lost signal to thread 11502
I20260709 13:16:55.053541 15768 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.024729 15416 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:55.056377 15900 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.456s	user 0.003s	sys 0.000s
W20260709 13:16:55.057610 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.103940 12112 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:55.104435 11515 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:55.060295 15751 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:55.084741 15886 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:55.120050 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 83206.2us
I20260709 13:16:55.097112 12085 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:55.097662 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:55.102607 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.123154 15336 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:55.077183 15754 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.124082 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.126057 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:55.127951 11572 debug-util.cc:398] Leaking SignalData structure 0x7b080033f280 after lost signal to thread 11575
W20260709 13:16:55.128319 11572 debug-util.cc:398] Leaking SignalData structure 0x7b080013fee0 after lost signal to thread 11576
W20260709 13:16:55.129555 11572 debug-util.cc:398] Leaking SignalData structure 0x7b08003defa0 after lost signal to thread 12147
I20260709 13:16:55.130713 15754 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:55.103585 15177 consensus_meta.cc:287] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.003s
I20260709 13:16:55.038275 15919 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:55.111969 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.035738 15688 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:55.121160 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:55.106537 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:55.118561 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79772.4us
I20260709 13:16:55.106006 15416 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.111110 15841 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.143337 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.144066 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.119232 15718 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:55.146471 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:55.105300 15874 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.154038 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:55.149147 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.157779 11545 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:55.141621 15691 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:55.166018 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:55.166177 15691 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:55.166371 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2422) took 1286 ms. Trace:
I20260709 13:16:55.167254 15691 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.134788 15904 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:55.139075 15909 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:55.174196 15909 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.174618 15909 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.177528 11399 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:55.139235 15892 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:16:55.146602 15886 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:16:55.146937 15776 consensus_queue.cc:1048] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:16:55.134917 15875 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:55.144531 15336 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:55.181028 15336 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:55.166769 11471 rpcz_store.cc:276] 0709 13:16:53.880412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.880908 (+   496us) service_pool.cc:224] Handling call
0709 13:16:53.881224 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.881809 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.881840 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.495938 (+614098us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:16:54.496531 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.592171 (+ 95640us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:54.592204 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.592269 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.055682 (+463413us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:55.166145 (+110463us) raft_consensus.cc:1685] finished
0709 13:16:55.166168 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.166319 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":706185984,"thread_start_us":84479,"threads_started":1,"wal-append.queue_time_us":103616}
I20260709 13:16:55.181418 15336 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:55.172912 15894 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:55.173187 12122 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:55.173271 15919 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.183418 15919 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.140100 15177 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:55.185863 11400 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:55.125983 15837 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.142879 15840 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.202538 12118 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:16:55.205097 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:55.203559 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:55.204093 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.205657 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:55.206605 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:55.207249 11545 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.159555 15692 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.220297 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:55.220876 11550 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:55.222477 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.223603 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.215164 15630 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.201835 15177 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:55.185863 15718 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:55.228724 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.238969 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.239490 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.240557 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.241197 11400 raft_consensus.cc:2368] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:16:55.244091 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.233237 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.237233 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:55.245981 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:55.219027 15466 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:55.249727 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.239632 11394 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:16:55.246784 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.247573 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:55.252269 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.273913 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.275597 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.276626 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.260704 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:55.277675 11542 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:55.271562 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:55.260943 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:55.281077 11400 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:55.281133 15765 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:55.281893 15765 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:55.282748 15765 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.282526 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:55.285270 11400 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:55.287226 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47335us
I20260709 13:16:55.287431 15769 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.287987 15769 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.290083 15769 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.311223 11400 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:16:55.230662 15177 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:55.219687 15751 consensus_queue.cc:1059] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:16:55.323709 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.326462 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.325484 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.330238 11473 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:55.315344 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:55.246025 15841 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.326421 11542 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:55.326048 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.331736 11476 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:55.331368 15841 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.332275 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.321902 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.333585 15841 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.333745 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:55.312498 15860 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:55.334975 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:55.335520 11545 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.338809 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.339018 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.339560 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.327251 11474 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:55.345600 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.345831 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.347147 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.348997 11470 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:55.349668 11470 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:55.350244 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.351125 11473 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:55.353013 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.260900 15776 raft_consensus.cc:2764] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:16:55.353662 11473 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:55.362416 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.354892 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.364065 11474 raft_consensus.cc:2376] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:55.350945 11470 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.369345 11470 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.355454 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.356529 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.371400 11476 raft_consensus.cc:2376] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6. Re-sending same reply.
I20260709 13:16:55.357486 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.372303 11471 raft_consensus.cc:2376] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10. Re-sending same reply.
I20260709 13:16:55.357776 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.373579 11468 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:55.374248 11469 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:55.238353 15852 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:16:55.225150 15910 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.376123 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.358512 11545 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:55.346822 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:55.359416 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.359398 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44146.5us
I20260709 13:16:55.359972 11550 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:16:55.229789 15630 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.463027 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.463843 11542 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:55.454311 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.464389 11476 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:55.464792 11476 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:55.455704 11545 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:16:55.455854 11400 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:55.458524 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.328991 15177 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.321635 15466 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:55.478132 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.481261 11550 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:55.474017 11476 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.484300 11542 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:16:55.377230 15899 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:55.378121 15765 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:55.491469 11545 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:16:55.493047 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 289536us
I20260709 13:16:55.493258 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.497905 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:55.498621 11545 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:55.461269 15860 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:55.502914 11550 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:55.500379 11545 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:55.501029 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:55.479647 15593 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:55.506950 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.508256 11470 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:16:55.510097 11470 pending_rounds.cc:85] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 5
I20260709 13:16:55.510353 11468 raft_consensus.cc:2410] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:55.510414 11470 pending_rounds.cc:107] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:16:55.510885 11470 raft_consensus.cc:2889] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:55.497155 15861 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:55.509059 15776 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:55.529448 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:55.533073 15926 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:16:55.533389 15921 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:55.509181 15765 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:55.526214 15861 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.534832 15765 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:55.535503 15925 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:16:55.536078 15466 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.536813 15630 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.537071 15861 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.490869 15718 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.544440 11546 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:55.548168 15910 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.531129 15852 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:55.567067 15768 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:55.569258 15807 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.569818 15807 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.513761 15593 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.573297 15807 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.531594 15776 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.508785 15860 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:55.577827 15887 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.224s	user 0.012s	sys 0.002s
I20260709 13:16:55.550289 15837 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.602588 15776 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:55.563825 15852 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.582654 15416 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.575126 15909 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.757764 15860 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.761215 15909 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.763363 15852 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.738536 15908 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:55.772529 15908 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.775844 15908 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.532271 11468 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 6.
I20260709 13:16:55.812790 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.815210 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.815780 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.816753 11470 raft_consensus.cc:2393] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:55.817538 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.817977 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.818717 11475 raft_consensus.cc:2410] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:55.819139 11468 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:55.828116 11395 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:16:55.827940 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.829061 11477 raft_consensus.cc:2410] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:16:55.815583 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.830418 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.831230 11549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.049s	sys 0.069s
W20260709 13:16:55.831825 11549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.049s	sys 0.069s
I20260709 13:16:55.832772 11543 raft_consensus.cc:2368] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11.
I20260709 13:16:55.814467 11401 raft_consensus.cc:1217] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-12.8]   Dedup: 10.7->[12.8-12.8]
I20260709 13:16:55.835040 11401 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:16:55.836808 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:55.840207 15416 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:55.851729 11402 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:55.856938 11396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.002s
W20260709 13:16:55.858493 11365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.000s
W20260709 13:16:55.859133 11365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.006s	sys 0.000s
I20260709 13:16:55.867545 11477 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:55.868074 11477 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:55.869449 11477 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:55.860908 11396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.003s	sys 0.002s
I20260709 13:16:55.878297 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:55.880353 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:55.889488 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.890390 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.891361 15466 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.892393 15466 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:55.894104 15336 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.712s	user 0.013s	sys 0.003s
I20260709 13:16:55.897266 15466 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.872485 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:55.901775 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:55.859504 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 671548us
I20260709 13:16:55.902951 11468 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:55.903980 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:55.904210 15718 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.904923 15718 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:55.906543 11514 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.578s (SENT)
I20260709 13:16:55.907383 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:55.908788 15955 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:55.915660 11547 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:55.917613 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.918591 11472 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:55.925997 11472 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:16:55.926721 11472 pending_rounds.cc:85] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 10
I20260709 13:16:55.894866 15336 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.929306 15336 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:55.927089 11472 pending_rounds.cc:107] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:16:55.930315 15593 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.931933 11472 raft_consensus.cc:2889] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:55.937019 15955 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.937810 15955 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:55.939808 11394 raft_consensus.cc:2368] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
W20260709 13:16:55.941679 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57390.2us
I20260709 13:16:55.948436 15765 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:16:55.952158 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.952967 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.960853 15955 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.963174 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.965081 11468 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:55.965909 15905 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:55.967859 15905 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:55.971065 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.969085 15718 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:55.968303 11477 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:55.984333 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:55.985097 11469 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.110455 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:55.988260 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.989145 11471 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:16:55.992005 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.992915 11474 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:16:55.992970 11300 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 10, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:55.994768 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:55.995433 11474 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:55.997761 15896 raft_consensus.cc:3055] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:55.998724 11469 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:55.999642 11300 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.004881 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:56.005857 11549 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:56.009174 11545 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:56.009912 15703 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:56.026451 15776 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.012436 15896 raft_consensus.cc:740] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:56.035759 11477 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:16:56.036206 11477 pending_rounds.cc:85] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 7
I20260709 13:16:56.036420 11477 pending_rounds.cc:107] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:16:56.036641 11477 raft_consensus.cc:2889] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:56.037249 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:56.053041 15776 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.058076 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1788) took 1032 ms. Trace:
W20260709 13:16:56.061717 15928 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.574s	user 0.004s	sys 0.013s
W20260709 13:16:56.063424 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 565234us
I20260709 13:16:56.067422 11468 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:56.069097 11476 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:56.069530 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51877.6us
I20260709 13:16:56.073220 15701 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:56.074909 15593 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:56.075438 15593 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:56.114213 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43487.4us
W20260709 13:16:56.116408 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.075244 15896 consensus_queue.cc:260] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.135422 11394 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:56.139565 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.143489 15899 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:56.145016 15899 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:56.147501 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 136612us
W20260709 13:16:56.151121 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.151324 15800 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:16:56.153991 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 250223us
I20260709 13:16:56.155613 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:56.156121 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:56.156455 11546 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:56.159518 15080 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:56.163858 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:56.149829 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.159541 15959 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:56.166904 15769 raft_consensus.cc:2764] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:16:56.167737 15959 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.168344 15959 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:56.170789 11401 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:16:56.198338 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.203917 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:56.207785 11402 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:56.219828 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:56.221801 15751 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:56.223086 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 73464.1us
I20260709 13:16:56.222954 11545 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.229169 11396 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:16:56.229918 11396 pending_rounds.cc:85] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 10
I20260709 13:16:56.230263 11396 pending_rounds.cc:107] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:16:56.230460 15800 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:56.231122 15800 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:56.231626 15800 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.232690 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 628) took 1034 ms. Trace:
I20260709 13:16:56.233148 11398 rpcz_store.cc:276] 0709 13:16:55.198435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.199459 (+  1024us) service_pool.cc:224] Handling call
0709 13:16:55.199858 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.218267 (+ 18409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.218296 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.218692 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.965577 (+746885us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18003fe1a0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:56.112794 (+147217us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:56.113186 (+   392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.119210 (+  6024us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.232200 (+112990us) raft_consensus.cc:1685] finished
0709 13:16:56.232224 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.232487 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65000,"raft.queue_time_us":41464,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":8416,"spinlock_wait_cycles":32462720,"thread_start_us":70045,"threads_started":1,"wal-append.queue_time_us":885840}
I20260709 13:16:56.235476 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.237843 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:56.239292 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.222910 11300 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:56.242204 15177 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.750s	user 0.000s	sys 0.021s
I20260709 13:16:56.242566 12123 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.242758 15177 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.243599 15919 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.243808 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.250914 11402 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:56.230630 11396 raft_consensus.cc:2889] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:56.224316 15915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.008s	sys 0.014s
I20260709 13:16:56.058583 11397 rpcz_store.cc:276] 0709 13:16:55.025380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.025516 (+   136us) service_pool.cc:224] Handling call
0709 13:16:55.025847 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.026076 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:55.131507 (+105431us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.361706 (+230199us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.056986 (+695280us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b180043f240. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8da682ea40 00007f8da68294e4 00007f8da6829455 00007f8da9911060 00007f8da9923716 00007f8da9934287 00007f8da9934208
0709 13:16:56.057350 (+   364us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.057460 (+   110us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:56.057574 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.057686 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.057901 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184844,"prepare.queue_time_us":230447,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":13018,"raft.queue_time_us":760138,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":8377,"spinlock_wait_cycles":29610240,"thread_start_us":285582,"threads_started":3,"wal-append.queue_time_us":190251}
I20260709 13:16:56.260844 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:56.251935 15177 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:56.264158 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97572.1us
I20260709 13:16:56.265105 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:56.257151 15915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.008s	sys 0.016s
I20260709 13:16:56.262231 11476 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:56.269255 11545 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.271703 15769 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:56.272044 15886 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.272269 15769 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.272814 15769 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:56.272622 15886 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.276546 15955 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:56.277802 15851 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:56.278203 15851 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:56.279315 15851 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.258126 15701 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:56.262879 15751 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:56.280814 15851 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:56.282199 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:56.280450 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 121510us
I20260709 13:16:56.286397 15701 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:56.288453 15896 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.295745 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1793) took 1197 ms. Trace:
I20260709 13:16:56.296833 11395 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.296483 15593 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.297904 15593 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:56.298100 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 2061) took 1035 ms. Trace:
I20260709 13:16:56.307582 11300 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 10 to 12, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.309540 11300 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.296201 11393 rpcz_store.cc:276] 0709 13:16:55.098357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.098478 (+   121us) service_pool.cc:224] Handling call
0709 13:16:55.119990 (+ 21512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.132509 (+ 12519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.132539 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.132962 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.961606 (+828644us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b1800439180. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:56.107873 (+146267us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:56.107951 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.292066 (+184115us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b540014a0d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:56.292120 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.295365 (+  3245us) raft_consensus.cc:1685] finished
0709 13:16:56.295386 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.295562 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3490,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":1370,"spinlock_wait_cycles":30367360,"thread_start_us":122640,"threads_started":1,"wal-append.queue_time_us":1084397}
I20260709 13:16:56.312317 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:56.312523 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2464) took 1018 ms. Trace:
I20260709 13:16:56.313395 15894 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:56.314718 15894 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:56.315219 15894 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:56.316232 15963 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:16:56.317435 15894 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.314714 15630 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:56.321587 15630 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.322063 15630 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:56.323307 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.324240 11299 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 216) took 40988 ms (client timeout 15000 ms). Trace:
I20260709 13:16:56.326089 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:56.298451 11395 rpcz_store.cc:276] 0709 13:16:55.262421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.829043 (+566622us) service_pool.cc:224] Handling call
0709 13:16:56.296598 (+467555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.297229 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.297894 (+   665us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540014a0d0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:56.298029 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074182016}
I20260709 13:16:56.320988 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:56.306631 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:56.341706 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.332507 11300 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 7 to 9, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.320991 15894 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:56.353029 15919 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:56.343380 15751 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:56.382105 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:56.383502 11544 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:16:56.362764 11546 raft_consensus.cc:2410] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:16:56.324779 11299 rpcz_store.cc:269] 0709 13:16:15.335567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.336015 (+   448us) service_pool.cc:224] Handling call
0709 13:16:56.323989 (+40987974us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:47.352889 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:47.353617 (+   728us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:47.353680 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:16:47.353722 (+    42us) tablet.cc:662] Decoding operations
0709 13:16:47.354329 (+   607us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:47.354400 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:47.354433 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:16:47.354863 (+   430us) tablet.cc:701] Row locks acquired
0709 13:16:47.354909 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:16:47.356079 (+  1170us) write_op.cc:270] Start()
0709 13:16:47.356229 (+   150us) write_op.cc:276] Timestamp: P: 1783603007355970 usec, L: 0
0709 13:16:47.356253 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:16:47.357091 (+   838us) log.cc:844] Serialized 1275 byte log entry
0709 13:16:49.002513 (+1645422us) op_driver.cc:464] REPLICATION: finished
0709 13:16:49.097408 (+ 94895us) write_op.cc:301] APPLY: starting
0709 13:16:49.097522 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:49.097982 (+   460us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:49.098100 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:49.098616 (+   516us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:49.098876 (+   260us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:16:49.098918 (+    42us) write_op.cc:312] APPLY: finished
0709 13:16:49.099193 (+   275us) log.cc:844] Serialized 35 byte log entry
0709 13:16:49.100059 (+   866us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.100144 (+    85us) write_op.cc:454] Released schema lock
0709 13:16:49.100196 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:49.100592 (+   396us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.100619 (+    27us) write_op.cc:454] Released schema lock
0709 13:16:49.326391 (+225772us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991632a 00007f8da989bf14 00007f8da989bec3 00007f8da989be38 00007f8da989bdf3 00007f8da989b0dc 00007f8da98c4e16 00007f8da98accbb 00007f8da989739d 00007f8da989ad75
Metrics: {"mutex_wait_us":4494726,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":94567,"apply.run_cpu_time_us":2372,"apply.run_wall_time_us":3165,"num_ops":3,"prepare.queue_time_us":611463,"prepare.run_cpu_time_us":28337,"prepare.run_wall_time_us":208008,"raft.queue_time_us":477027,"raft.run_cpu_time_us":21442,"raft.run_wall_time_us":348558,"replication_time_us":1646028,"spinlock_wait_cycles":3766656,"thread_start_us":764326,"threads_started":4,"wal-append.queue_time_us":307647,"wal-append.run_cpu_time_us":185769,"wal-append.run_wall_time_us":2434082}]]}
I20260709 13:16:56.387452 15973 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:56.388852 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.363250 15975 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:56.363917 15886 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:56.364467 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:56.365113 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:56.312958 11551 rpcz_store.cc:276] 0709 13:16:55.194102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.203973 (+  9871us) service_pool.cc:224] Handling call
0709 13:16:55.204233 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.204413 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:56.061833 (+857420us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18003cc040. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8dad3113c7 00007f8da99239a7 00007f8da99330bf
0709 13:16:56.062071 (+   238us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.121795 (+ 59724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.124765 (+  2970us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:56.171986 (+ 47221us) spinlock_profiling.cc:243] Waited 926 us on lock 0x7b18003cc040. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:16:56.181625 (+  9639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.201744 (+ 20119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.212353 (+ 10609us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28109,"prepare.queue_time_us":158629,"prepare.run_cpu_time_us":8675,"prepare.run_wall_time_us":93771,"raft.queue_time_us":4740,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":8173,"spinlock_wait_cycles":13948544,"thread_start_us":291734,"threads_started":2,"wal-append.queue_time_us":69003}
I20260709 13:16:56.366083 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:56.369571 15959 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:56.372206 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.375533 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.376534 11543 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:56.380467 15974 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:56.397539 15974 consensus_queue.cc:1243] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:16:56.398131 15974 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:56.398516 15974 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:56.398874 15959 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.399657 15974 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.401000 15974 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.381227 12113 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:56.382961 15919 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.404568 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:56.405592 15919 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:56.359380 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47181.4us
I20260709 13:16:56.406126 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:56.406728 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.407222 15851 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:56.408371 15701 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:56.408658 15975 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:56.408974 15701 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:56.409401 15416 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:56.409219 15915 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:56.410027 15701 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.409986 15416 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:56.411794 15416 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.414680 11542 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.415843 15919 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.416435 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:56.417948 15336 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:56.388365 15973 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:16:56.392416 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85923.2us
I20260709 13:16:56.416613 15919 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.421391 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:56.420071 15973 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.396862 15807 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:56.423179 15807 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.423593 15807 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.416464 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:56.425406 11548 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:16:56.425729 11471 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:56.426391 15955 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:56.428632 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.414680 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.444168 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:56.445892 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.448007 11471 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:16:56.446604 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.450488 11470 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:56.451124 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.448487 11471 pending_rounds.cc:85] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 7
I20260709 13:16:56.453739 11471 pending_rounds.cc:107] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:16:56.454057 11471 raft_consensus.cc:2889] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:56.441581 11468 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:56.455031 11468 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:56.444272 15886 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:56.456972 15886 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.460904 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.443881 15955 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:56.463395 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:56.463660 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.464233 11399 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:56.464651 11442 leader_election.cc:400] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:16:56.464983 15840 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:56.465003 11442 leader_election.cc:403] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:56.468799 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.472105 11542 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.409816 15975 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.471858 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.471890 11300 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.471776 15638 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:16:56.480065 11468 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.481173 11468 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:56.481000 15769 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:56.483106 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.467489 15886 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.475215 15336 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:56.486733 15336 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:56.487180 15336 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.489571 11399 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:56.491134 15974 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:56.491460 11393 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.491567 15974 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:56.493865 15974 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.495572 15974 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:56.495819 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.505383 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.463413 15955 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:56.497120 15840 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:16:56.500703 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.502411 11393 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.439836 15800 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.503264 15919 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.504534 11396 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.525692 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:56.526152 15630 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.528314 15705 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:56.528843 15705 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.529248 15705 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:56.503785 11300 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.530557 15692 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 13:16:56.531708 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 117236us
I20260709 13:16:56.531124 11300 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 6 to 9, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 9 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.532667 15955 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.533161 11300 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:56.508852 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.535760 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:56.534828 11300 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.557897 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:56.557817 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.560765 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:56.565384 11398 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:56.585158 11396 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:56.540623 15919 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:56.521847 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:56.522202 15800 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:56.522719 15703 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:56.524137 15920 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:56.600404 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:56.590683 11393 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:56.515977 11470 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:56.597290 11300 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 10 to 12, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 12 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:56.606112 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 177580us
I20260709 13:16:56.607853 11545 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:56.609159 12115 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:56.610707 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:56.611502 12115 raft_consensus.cc:2393] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:16:56.613443 12115 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.548509 15915 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:16:56.615607 11300 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 7 to 8, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.649221 11397 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:56.651140 15849 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:16:56.652868 15769 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.649639 11300 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 5 to 8, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.654307 12122 raft_consensus.cc:1240] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.670003 15920 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:56.670471 15920 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:56.670909 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.671949 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.672716 11474 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:56.675441 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.668805 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:56.677032 11300 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 6 to 13, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.563375 15416 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:56.690945 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 84207.2us
I20260709 13:16:56.691167 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:56.691706 12122 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:56.693203 11442 leader_election.cc:400] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:56.695201 15896 raft_consensus.cc:2764] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:16:56.652789 15807 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.676739 15919 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.720685 15918 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.723240 15703 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:56.724967 15920 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:56.748409 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 53528.4us
I20260709 13:16:56.727403 11398 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:56.776129 11468 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:16:56.781822 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.782333 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.783078 15769 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:16:56.790372 11402 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:56.791639 12118 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.790537 11471 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:56.796660 12122 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:56.798364 11470 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
W20260709 13:16:56.803411 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 109511us
W20260709 13:16:56.800623 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 296032us
I20260709 13:16:56.806002 11300 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 3 to 4, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 4 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:16:56.809267 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:56.818774 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.798101 15177 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.791244 15918 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.827941 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:56.832568 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 85764.7us
I20260709 13:16:56.833446 15807 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.836246 15776 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:56.832631 15920 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:56.840857 11470 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:16:56.854512 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.856765 15701 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:56.823741 15886 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:56.868274 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.869486 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.872820 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.859869 15630 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.883936 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:56.885075 11477 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:16:56.831631 15177 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:56.896945 15807 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:56.898965 15776 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:56.899413 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:56.901002 15776 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:56.901584 15776 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:56.905884 15692 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:56.906379 15692 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:56.907383 15692 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.908627 15692 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.908691 15919 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:56.909554 15919 raft_consensus.cc:2764] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:16:56.912312 15765 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:56.916262 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:56.917773 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:16:56.918864 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.922430 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82768.9us
I20260709 13:16:56.926935 11470 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:56.859026 15896 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:16:56.943699 15983 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:16:56.963719 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:56.964659 11468 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.910936 15919 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:56.984635 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:56.994596 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:56.943270 15630 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.998718 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.902735 15776 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:56.936105 15765 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 13:16:56.994758 15336 consensus_meta.cc:287] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.006s
W20260709 13:16:57.024430 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61177.9us
I20260709 13:16:56.990571 15416 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:16:57.032238 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.007193 15840 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:16:56.955288 15705 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.114953 15705 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.100049 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 632) took 1801 ms. Trace:
I20260709 13:16:57.115931 11400 rpcz_store.cc:276] 0709 13:16:55.207680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.474196 (+266516us) service_pool.cc:224] Handling call
0709 13:16:55.474511 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.215538 (+741027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.219500 (+  3962us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.260994 (+ 41494us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.562957 (+301963us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:56.563017 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.589454 (+ 26437us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b540035ded4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:56.589520 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.997727 (+408207us) raft_consensus.cc:1685] finished
0709 13:16:56.997744 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.008805 (+ 11061us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":574719,"raft.queue_time_us":648850,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156,"spinlock_wait_cycles":38009600,"thread_start_us":311373,"threads_started":2,"wal-append.queue_time_us":514618}
W20260709 13:16:57.110066 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.095551 15983 log.cc:927] Time spent T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:16:57.119674 15995 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:16:57.122550 15701 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:16:57.123337 15997 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:57.124656 15701 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:57.125131 15701 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:57.125598 15701 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:57.127801 15918 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.110491 15703 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.144817 11468 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:57.148072 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:57.157061 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 327898us
I20260709 13:16:57.159967 11474 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:57.163203 11471 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:57.176678 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:57.140595 15840 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:57.093688 15851 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.026332 15630 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.198925 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:57.198616 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.048885 15336 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.203806 11475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.022s	sys 0.039s
W20260709 13:16:57.213610 11475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.023s	sys 0.039s
I20260709 13:16:57.213550 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:57.184903 15807 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:57.153527 15919 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.254308 11474 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:57.162633 15703 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.245894 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47417.6us
W20260709 13:16:57.259508 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 111539us
I20260709 13:16:57.260739 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:57.211859 15751 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:57.213402 15896 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.266424 11399 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.214681 15840 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.267133 11468 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:16:57.269711 15993 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:16:57.269933 15998 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.271188 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:57.272413 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:57.275584 15768 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:57.275959 11473 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:16:57.276610 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 358929us
I20260709 13:16:57.335618 11469 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:16:57.353101 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 92084.9us
I20260709 13:16:57.267498 15896 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:57.357380 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:57.361589 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:57.362701 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 91693.6us
I20260709 13:16:57.354391 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:57.365912 11473 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:57.292644 15177 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.288671 15768 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:57.370278 15999 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.371414 16000 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:16:57.373502 15751 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:57.376415 15703 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.346431 15886 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.376943 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.378173 15630 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:57.385161 15982 log.cc:927] Time spent T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.451s	user 0.002s	sys 0.000s
I20260709 13:16:57.346134 15840 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:57.392815 11547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.000s	sys 0.024s
W20260709 13:16:57.393196 11542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.004s	sys 0.020s
I20260709 13:16:57.402019 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:57.419660 11542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.005s	sys 0.020s
I20260709 13:16:57.404517 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:57.407105 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.419528 11547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.000s	sys 0.025s
I20260709 13:16:57.407167 15751 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:57.423291 15638 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:57.423748 15638 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:57.424125 15638 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:57.388787 15920 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:16:57.419528 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:57.435065 11471 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:57.447726 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 86245.3us
W20260709 13:16:57.408659 15955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.019s	sys 0.032s
I20260709 13:16:57.429368 11474 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:16:57.414350 15718 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:57.462713 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 113531us
I20260709 13:16:57.406391 15593 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:57.468470 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.469132 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:57.453583 15955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.020s	sys 0.032s
I20260709 13:16:57.425617 16002 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:57.471032 11475 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:57.491694 11471 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:57.492645 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:57.494837 11475 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:57.478945 11469 pending_rounds.cc:85] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 10
I20260709 13:16:57.500427 11469 pending_rounds.cc:107] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:16:57.500738 11469 raft_consensus.cc:2889] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:57.501870 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 142848us
I20260709 13:16:57.461256 15886 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:57.401235 15998 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.526881 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:57.535070 11441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.009s	sys 0.015s
W20260709 13:16:57.535125 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.526415 15080 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:16:57.537161 11441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.009s	sys 0.016s
I20260709 13:16:57.406088 15999 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:57.538806 11515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.018s	sys 0.030s
W20260709 13:16:57.540421 11515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.019s	sys 0.030s
W20260709 13:16:57.540787 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 619211us
W20260709 13:16:57.541041 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72034.5us
I20260709 13:16:57.480474 16001 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:16:57.543118 11517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.014s	sys 0.029s
W20260709 13:16:57.544315 11517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.014s	sys 0.029s
I20260709 13:16:57.534765 15955 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.534279 16002 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:57.561673 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:57.555330 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:57.556674 11300 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.040s	sys 0.058s
W20260709 13:16:57.565232 11300 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.041s	sys 0.058s
I20260709 13:16:57.572659 16006 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:57.579025 16015 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:16:57.591606 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:57.593680 16013 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:16:57.595476 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45930.2us
I20260709 13:16:57.596451 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:57.600842 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 108712us
I20260709 13:16:57.536505 15886 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.504884 15896 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.612185 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50617.4us
I20260709 13:16:57.500563 15768 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.599025 15999 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.599581 15919 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:57.638149 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.606403 16001 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.604990 15807 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:57.657779 11476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.002s	sys 0.017s
W20260709 13:16:57.658160 11476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.002s	sys 0.017s
W20260709 13:16:57.658946 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.668938 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:57.672979 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.673771 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.647243 15896 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.611603 16006 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:16:57.645973 15336 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:57.705605 11547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2480) took 1370 ms. Trace:
I20260709 13:16:57.602710 16002 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.706432 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:57.721797 11549 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.661430 15800 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:16:57.706143 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 244248us
I20260709 13:16:57.644997 15776 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:16:57.670644 16015 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:57.720458 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:16:57.727362 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 90526.2us
I20260709 13:16:57.728333 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.696365 15765 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.696149 15593 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.694511 15769 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.713147 11547 rpcz_store.cc:276] 0709 13:16:56.322441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.327045 (+  4604us) service_pool.cc:224] Handling call
0709 13:16:56.327525 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.327889 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:56.797576 (+469687us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.686339 (+888763us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.692958 (+  6619us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:57.693002 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.693075 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.693225 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6176,"prepare.queue_time_us":51353,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":172124,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":157,"thread_start_us":1344221,"threads_started":3,"wal-append.queue_time_us":737962}
I20260709 13:16:57.801841 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:57.705996 15718 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:57.753856 15840 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:57.820580 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 224201us
W20260709 13:16:57.772266 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 130765us
W20260709 13:16:57.825110 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 102279us
W20260709 13:16:57.813156 16011 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:16:57.825845 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:57.806149 15955 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:57.771893 15800 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.892175 15800 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:57.766973 16015 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.838133 16014 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 13:16:57.813474 15807 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.838939 16024 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:16:57.902810 16043 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:16:57.839426 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 643) took 1235 ms. Trace:
I20260709 13:16:57.903955 11393 rpcz_store.cc:276] 0709 13:16:56.603943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.681799 (+ 77856us) service_pool.cc:224] Handling call
0709 13:16:56.711599 (+ 29800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.279476 (+567877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.279513 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:57.279985 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.658650 (+378665us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:57.658689 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.658753 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.839156 (+180403us) raft_consensus.cc:1685] finished
0709 13:16:57.839176 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.839364 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20939,"raft.queue_time_us":25044,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2386,"thread_start_us":737477,"threads_started":2,"wal-append.queue_time_us":461504}
W20260709 13:16:57.839682 16010 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:16:57.840176 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:57.908272 11544 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:16:57.840605 16017 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.831250 15703 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:57.846448 11543 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:57.910653 16017 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.911387 15703 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.913317 15703 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:57.914064 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2490) took 1366 ms. Trace:
I20260709 13:16:57.914430 11542 rpcz_store.cc:276] 0709 13:16:56.548069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.548233 (+   164us) service_pool.cc:224] Handling call
0709 13:16:56.595582 (+ 47349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:56.782210 (+186628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.782256 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:56.800045 (+ 17789us) log.cc:844] Serialized 52 byte log entry
0709 13:16:57.421096 (+621051us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:57.453833 (+ 32737us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.477815 (+ 23982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.477915 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.913834 (+435919us) raft_consensus.cc:1685] finished
0709 13:16:57.913868 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.914006 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12820,"prepare.queue_time_us":40292,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":8308,"raft.queue_time_us":222280,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":7194,"spinlock_wait_cycles":185088,"thread_start_us":574424,"threads_started":1,"wal-append.queue_time_us":724085}
W20260709 13:16:57.847671 16019 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:16:57.799383 15765 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:57.917105 15701 consensus_meta.cc:287] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.791s	user 0.012s	sys 0.000s
I20260709 13:16:57.917579 15701 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.918756 15765 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.919580 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 644) took 1016 ms. Trace:
I20260709 13:16:57.919931 11397 rpcz_store.cc:276] 0709 13:16:56.903409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.905331 (+  1922us) service_pool.cc:224] Handling call
0709 13:16:56.905778 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.941610 (+ 35832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.941657 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.953640 (+ 11983us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.377934 (+424294us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b48000d0458. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8dae842aa1 00007f8dae839b37 00007f8dae8399ca 00007f8dae93126f 00007f8dae930f37 00007f8dae94555c 00007f8dae945508 00007f8dae945488 00007f8dae945443 00007f8dae9446ec 00007f8dad86ea96
0709 13:16:57.471528 (+ 93594us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:57.471585 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.471674 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.919307 (+447633us) raft_consensus.cc:1685] finished
0709 13:16:57.919326 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.919525 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106113,"raft.queue_time_us":36023,"raft.run_cpu_time_us":3501,"raft.run_wall_time_us":433719,"spinlock_wait_cycles":5666176,"thread_start_us":451385,"threads_started":1,"wal-append.queue_time_us":626080}
I20260709 13:16:57.856171 16033 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:57.863921 11475 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:57.921756 16033 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:57.922857 15701 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:57.923178 15973 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.922755 16033 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:57.858268 16008 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 13:16:57.866161 16012 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:16:57.925244 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:16:57.872191 12085 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.3:46781 timed out after 1.742s (SENT) [suppressed 22 similar messages]
I20260709 13:16:57.851887 15998 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:57.874831 16037 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:57.887382 16023 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:16:57.823827 15851 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:57.833570 15769 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.895638 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:57.920997 12123 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:57.707715 16001 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:57.870913 11473 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:16:57.922652 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 645) took 1001 ms. Trace:
I20260709 13:16:57.935995 11473 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:57.884040 15886 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.940168 16004 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.940639 16004 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:57.942186 16004 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:57.942862 15840 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:57.943090 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47510.7us
I20260709 13:16:57.944346 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.949570 11549 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:57.934746 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:57.936023 11402 rpcz_store.cc:276] 0709 13:16:56.920823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.920965 (+   142us) service_pool.cc:224] Handling call
0709 13:16:56.934049 (+ 13084us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:56.983267 (+ 49218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.983307 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:57.018280 (+ 34973us) log.cc:844] Serialized 52 byte log entry
0709 13:16:57.533588 (+515308us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:57.538932 (+  5344us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.900442 (+361510us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:57.922112 (+ 21670us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b180043f5a0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8da682ea40 00007f8da68294e4 00007f8da6829455 00007f8da9911060 00007f8da9923716 00007f8da9934287 00007f8da9934208
0709 13:16:57.922338 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.922412 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.922450 (+    38us) raft_consensus.cc:1685] finished
0709 13:16:57.922463 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.922598 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32661,"prepare.queue_time_us":5316,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1201,"raft.queue_time_us":411440,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":21109,"spinlock_wait_cycles":20129920,"thread_start_us":424010,"threads_started":1,"wal-append.queue_time_us":712544}
I20260709 13:16:57.925729 15593 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.927500 16043 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.953322 15688 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:57.962383 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 234125us
I20260709 13:16:57.932533 16037 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:16:57.969832 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44702.9us
I20260709 13:16:57.969949 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:57.953930 15886 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:57.983966 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.986974 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 114709us
W20260709 13:16:57.987491 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.985337 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.966631 15918 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:57.988870 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:57.989638 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:57.990118 16043 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.993337 16029 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:16:57.996260 15099 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.997340 15955 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.990195 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 773) took 1068 ms. Trace:
I20260709 13:16:57.999939 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.008239 11394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 642) took 1416 ms. Trace:
I20260709 13:16:58.003618 11442 leader_election.cc:400] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11.
I20260709 13:16:57.945616 15998 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.997635 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:57.968250 15416 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.975193 16033 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:58.050652 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 56620.9us
W20260709 13:16:58.001711 16018 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:16:58.069830 11473 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:58.070569 15416 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.071184 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.072651 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.072929 15416 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.074177 11398 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
W20260709 13:16:57.998327 15638 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.003s
I20260709 13:16:58.075676 11396 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:16:58.076313 11396 pending_rounds.cc:85] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 6
I20260709 13:16:58.076608 11396 pending_rounds.cc:107] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 9.7
W20260709 13:16:58.005085 15751 consensus_meta.cc:287] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.000s
I20260709 13:16:57.999413 16037 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.060839 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.061348 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.061908 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2464) took 1390 ms. Trace:
I20260709 13:16:58.064055 15768 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.989780 16002 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.008183 15769 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.006052 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:58.076876 11396 raft_consensus.cc:2889] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:58.078511 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.048079 11472 rpcz_store.cc:276] 0709 13:16:56.922015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.127731 (+205716us) service_pool.cc:224] Handling call
0709 13:16:57.128136 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.128501 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.128537 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:57.129092 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.534485 (+405393us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:57.534545 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.534673 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.989941 (+455268us) raft_consensus.cc:1685] finished
0709 13:16:57.989960 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.990120 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69866,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15173,"thread_start_us":396729,"threads_started":1,"wal-append.queue_time_us":567267}
I20260709 13:16:57.999415 16001 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:16:58.080298 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 82773us
I20260709 13:16:58.080818 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 772) took 1160 ms. Trace:
W20260709 13:16:58.085755 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 82266.2us
I20260709 13:16:58.048754 11394 rpcz_store.cc:276] 0709 13:16:56.591770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.596922 (+  5152us) service_pool.cc:224] Handling call
0709 13:16:56.648817 (+ 51895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.791154 (+142337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.791209 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.799782 (+  8573us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.291298 (+491516us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:57.291351 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.291440 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:58.008028 (+716588us) raft_consensus.cc:1685] finished
0709 13:16:58.008046 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.008183 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103694,"raft.queue_time_us":120887,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":9511,"thread_start_us":459701,"threads_started":1,"wal-append.queue_time_us":709613}
I20260709 13:16:57.997797 15705 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:16:58.082396 11476 rpcz_store.cc:276] 0709 13:16:56.671281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.671609 (+   328us) service_pool.cc:224] Handling call
0709 13:16:56.671953 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.980099 (+308146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.980326 (+   227us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.981973 (+  1647us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.658499 (+676526us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:57.658540 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.658610 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.992365 (+333755us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:58.061592 (+ 69227us) raft_consensus.cc:1685] finished
0709 13:16:58.061612 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.061833 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":238185,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"spinlock_wait_cycles":9213824,"thread_start_us":798584,"threads_started":2,"wal-append.queue_time_us":840267}
I20260709 13:16:58.087432 11395 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:58.093025 11470 rpcz_store.cc:276] 0709 13:16:56.920448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.942580 (+ 22132us) service_pool.cc:224] Handling call
0709 13:16:56.980535 (+ 37955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.980993 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.981032 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.981594 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:16:57.492295 (+510701us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:57.492354 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.492443 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:58.001002 (+508559us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:58.076326 (+ 75324us) raft_consensus.cc:1685] finished
0709 13:16:58.076357 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.080764 (+  4407us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":159338,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":101,"spinlock_wait_cycles":30715648,"thread_start_us":471896,"threads_started":1,"wal-append.queue_time_us":643961}
I20260709 13:16:58.098831 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:58.100024 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.101097 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:58.079197 15998 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.988971 15718 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.111337 15701 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:58.111534 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 105582us
I20260709 13:16:58.112412 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:58.113173 15701 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:58.113593 15701 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.113653 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:58.113996 15701 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:58.088670 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:57.967935 15593 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:57.999543 16017 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.059108 16051 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:16:58.089385 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:58.058709 15918 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.085493 15416 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:58.125878 16051 consensus_queue.cc:1243] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:58.144127 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.134157 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.169373 11399 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:58.060331 15955 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.130641 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:16:58.171466 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56640.5us
I20260709 13:16:58.057934 16043 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:58.174626 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44042.6us
I20260709 13:16:58.177105 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:58.140436 15886 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:58.081686 15751 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.186098 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.135802 15998 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.079613 15638 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.128943 16013 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:58.140513 16002 raft_consensus.cc:2764] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:16:58.203450 15906 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:58.204028 15906 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.136502 16015 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.145406 15765 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:58.188719 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:58.128648 15840 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:58.135141 16017 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:58.187090 15416 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.228703 11397 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:58.133132 15997 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:16:58.193159 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:58.193748 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.191623 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:16:58.196105 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.231580 11549 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:16:58.178038 16037 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:58.232649 11546 raft_consensus.cc:2393] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:16:58.200119 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.186115 16035 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.193387 15692 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:58.238940 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 47392us
I20260709 13:16:58.116312 16001 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.189599 16020 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:58.192967 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.129042 16046 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:16:58.199716 11398 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:58.234886 11365 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.201537 16013 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.255616 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.198704 15688 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:16:58.226336 15765 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.238387 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:58.246028 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.256902 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.276084 11542 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.246940 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.247633 11543 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:58.228487 15776 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.230273 16006 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:58.243525 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54883.9us
W20260709 13:16:58.286222 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47953.8us
I20260709 13:16:58.277366 11545 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:58.258308 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.280715 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.236006 15894 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.284788 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:58.255051 15959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.003s	sys 0.012s
I20260709 13:16:58.289312 11547 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:58.291347 11394 raft_consensus.cc:2393] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:58.244163 15915 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:58.296043 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.291898 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.302181 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.302475 11544 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:58.242103 16015 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:58.247977 16020 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.274791 16013 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:58.305279 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.298532 16021 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.317976 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:58.294111 15959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.003s	sys 0.013s
I20260709 13:16:58.280200 15765 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.348173 16036 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.348711 16036 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:58.320132 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61949.7us
I20260709 13:16:58.350870 16036 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.351056 15638 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.334053 15915 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.320824 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.354524 15915 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.355463 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.356714 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.357122 11394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:58.357803 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.256364 15896 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:58.227702 15906 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.241765 16035 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.280385 15975 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:58.293262 15807 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.340258 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.291114 15776 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.343824 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.359390 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.357836 11394 raft_consensus.cc:606] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.363695 11472 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:16:58.364362 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.371124 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.318630 15894 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:58.423815 11477 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:58.373590 11469 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:58.331698 16028 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:58.366514 15896 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:58.375243 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:58.240916 16017 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.375880 11473 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:58.376225 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.454245 11470 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:58.376592 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.336261 16020 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.339730 16003 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:58.416307 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.379782 11544 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:16:58.416903 11475 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:58.423362 11394 raft_consensus.cc:614] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.464404 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:58.294430 16006 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.328370 16015 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.454583 11471 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:16:58.459630 16019 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:16:58.336639 15692 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:16:58.473197 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.336990 15989 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:16:58.258586 15918 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.428259 16028 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:16:58.458434 16055 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:16:58.427946 15840 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:58.358201 15416 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.484432 11468 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:58.460130 15975 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.466167 15336 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:58.424002 15894 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.475553 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.516803 16001 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.517946 15894 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.520839 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.522061 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.523213 16025 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.523769 16025 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.480059 15593 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.516841 11398 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:16:58.502472 11470 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:58.483363 16003 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:58.506737 16028 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.511077 16033 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:58.514355 15336 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.538167 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:16:58.540109 15336 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.514803 15718 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:58.542114 16054 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:16:58.542747 16032 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.170s
I20260709 13:16:58.544162 16032 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:58.544595 16032 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:58.545603 16032 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:58.547696 16053 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:16:58.548631 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:58.549162 16058 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
W20260709 13:16:58.550686 16059 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:16:58.551703 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:58.552481 11473 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.495390 15751 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.515769 15765 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.493357 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.480473 16015 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:58.525724 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.495273 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.569849 11471 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:58.489933 16017 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.372253 16021 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.489144 15955 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.556022 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.575106 11476 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:58.515462 15975 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:58.534360 15851 consensus_meta.cc:287] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.004s
W20260709 13:16:58.574285 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 248154us
I20260709 13:16:58.499202 15840 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.606190 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:16:58.607520 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 245367us
W20260709 13:16:58.607784 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57052.3us
I20260709 13:16:58.606979 11551 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:58.577080 16032 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:58.610841 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.614933 11471 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:16:58.611469 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 93189.3us
I20260709 13:16:58.620447 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.602625 15851 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:58.597266 16069 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:58.527561 15896 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.629853 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.606323 15099 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:16:58.635860 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:58.598979 15336 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:16:58.606252 15688 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:16:58.640416 11543 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:16:58.639727 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.608546 15705 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:16:58.644152 11550 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:58.647431 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.648025 11471 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:58.644500 11473 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:58.649742 16025 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.650524 16070 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:58.650946 16070 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.651283 16070 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:58.654323 15807 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:58.654484 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.634045 16069 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:58.656631 16069 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:58.657640 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.646536 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:58.662577 11542 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:16:58.659466 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:58.637478 16004 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:58.675369 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.671162 15336 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:58.676769 15336 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.677114 15336 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:58.677407 15840 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.677323 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.678635 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.679064 16033 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:58.679726 16074 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:16:58.681365 16075 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.671362 15896 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:58.642515 16003 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.672394 15776 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.685832 16004 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:16:58.688328 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.686180 15894 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:58.695724 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:16:58.687062 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.703249 12114 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:58.706400 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:58.707257 11542 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.693264 16033 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.710494 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.707604 15894 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:58.698925 15765 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.716600 15894 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:58.725500 16064 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:16:58.721518 15638 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.706343 16004 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:58.726646 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.729200 11550 raft_consensus.cc:2410] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:58.729681 11402 raft_consensus.cc:606] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:58.703400 15807 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:16:58.719520 16035 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.710228 16006 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.720736 15776 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.730472 16075 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.740121 16075 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:58.740722 16060 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:16:58.742251 16075 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.734026 15638 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.744513 11544 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:16:58.752104 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:58.752522 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.756527 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.757210 11394 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:16:58.757630 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:58.758275 11400 raft_consensus.cc:3060] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:58.762050 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.762176 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:16:58.762940 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:58.769807 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.770694 11542 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:16:58.775624 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.775609 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.771924 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:16:58.747454 16071 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:16:58.780936 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 163294us
I20260709 13:16:58.781008 11544 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:16:58.784607 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:58.735273 15918 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:16:58.788393 15701 consensus_meta.cc:287] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.000s
I20260709 13:16:58.789611 16021 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:58.790068 15768 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.698s	user 0.007s	sys 0.000s
I20260709 13:16:58.781831 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:58.794142 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.749816 15638 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.784828 12085 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:58.779322 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:58.800907 11542 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:16:58.788843 15701 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.803068 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:58.809832 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.807324 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.812258 11468 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 7.
I20260709 13:16:58.782192 16047 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:58.808337 11544 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:58.783955 16006 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.781036 15920 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:58.813431 15765 raft_consensus.cc:2804] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:58.795508 15955 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.817914 11397 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:58.791546 16017 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.835237 16017 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.836187 11397 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:58.819532 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.840333 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:58.840552 15991 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:58.841075 11468 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:58.841140 15991 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:58.841781 15991 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.840385 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:16:58.843923 16087 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:16:58.845036 11285 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b3abf4b5ee724203ad1226f5c46bfe46 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:16:58.845697 11285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 62753.9us
I20260709 13:16:58.847402 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:16:58.750718 15776 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.845839 15701 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.813587 16047 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.837544 16017 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:58.858376 16047 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:58.857647 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.872185 15920 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:58.872283 15765 raft_consensus.cc:697] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:58.873277 15920 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.873514 15765 consensus_queue.cc:237] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:58.875141 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45160.5us
I20260709 13:16:58.828219 16006 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.857091 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:58.886133 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:58.896152 11547 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:58.896687 11547 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:58.898968 11544 raft_consensus.cc:1240] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:58.902654 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.902810 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.862470 15593 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:16:58.916225 11547 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.927161 11547 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:58.929546 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.893206 15701 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:58.931584 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:58.938305 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:58.899854 16033 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:58.921417 15955 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:58.907707 16080 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:58.942811 11400 raft_consensus.cc:2468] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:16:58.886914 15768 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:16:58.924494 16046 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.939901 15751 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.945478 16080 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.959872 12119 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:58.940670 15894 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.968354 15894 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.946168 15701 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.953603 16046 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.964421 16080 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:59.016479 16046 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.016744 16047 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:58.990545 16092 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:16:59.023192 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:16:59.025424 16065 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.681s	user 0.006s	sys 0.000s
W20260709 13:16:58.998548 16057 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.540s	user 0.000s	sys 0.002s
I20260709 13:16:58.999127 16070 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.019614 16047 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.029013 16070 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:58.999927 16021 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:59.001255 15701 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.002683 16015 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.003261 15336 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:58.991094 16006 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:59.005872 16020 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 13:16:58.990779 15991 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:58.998725 16004 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:59.008164 16013 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.003s
W20260709 13:16:59.009382 16056 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.724s	user 0.001s	sys 0.001s
I20260709 13:16:59.010347 11546 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:59.031697 11542 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:59.033941 11551 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:16:59.051909 11547 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:16:59.079599 11547 pending_rounds.cc:85] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 7
I20260709 13:16:59.080003 11547 pending_rounds.cc:107] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:16:59.080389 11547 raft_consensus.cc:2889] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:59.085017 16069 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.087888 11549 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:16:59.088285 11549 pending_rounds.cc:85] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 10
I20260709 13:16:59.088513 11549 pending_rounds.cc:107] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:16:59.088752 11549 raft_consensus.cc:2889] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:59.088773 16069 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.090425 15336 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.078752 16013 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.091845 12114 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:59.093842 16013 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.094887 11394 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:59.096115 16004 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.077445 16020 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.099671 11550 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:59.100495 11399 raft_consensus.cc:2368] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:16:59.059856 16001 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:59.110066 16020 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:16:59.131402 16040 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:16:59.147430 16006 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.148027 16006 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.149722 16021 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:59.150279 16006 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.151527 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:59.156677 15886 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.157202 15886 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:59.160558 16063 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.700s	user 0.000s	sys 0.002s
W20260709 13:16:59.162482 16093 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:16:59.163187 16080 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.127408 16015 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.165148 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55076 (request call id 628) took 1469 ms. Trace:
I20260709 13:16:59.165545 12113 rpcz_store.cc:276] 0709 13:16:57.695874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.707175 (+ 11301us) service_pool.cc:224] Handling call
0709 13:16:57.730722 (+ 23547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.929411 (+198689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:57.929447 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:57.938150 (+  8703us) log.cc:844] Serialized 26 byte log entry
0709 13:16:58.102885 (+164735us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b1800122f20. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:16:58.201686 (+ 98801us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:58.201725 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.355346 (+153621us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54001d0bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:58.355395 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.159968 (+804573us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:16:59.164918 (+  4950us) raft_consensus.cc:1685] finished
0709 13:16:59.164939 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.165065 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17217,"raft.queue_time_us":8073,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":8820,"spinlock_wait_cycles":467267456,"thread_start_us":98628,"threads_started":1,"wal-append.queue_time_us":441379}
I20260709 13:16:59.150830 16006 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.112883 16001 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.150305 16021 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.173719 16001 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.174729 16043 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.175002 16021 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.175379 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:59.171058 16006 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.183349 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:59.183938 12113 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:16:59.191256 15886 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.206275 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:16:59.231365 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.257792 11477 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:59.258215 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.257845 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.259488 11471 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:59.259577 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.260216 11398 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:16:59.262217 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.262863 11394 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.264935 11472 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:59.266340 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.267693 11476 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:16:59.272580 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:59.262130 16080 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:16:59.275889 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.212045 16040 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:16:59.265092 16006 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.279253 15099 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:59.279299 11473 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:59.280150 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.292153 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.268577 16015 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:16:59.295523 16043 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:59.295955 16043 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:16:59.296814 16043 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.298079 16043 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:59.282438 15974 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:59.289878 16112 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:59.311102 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:16:59.316160 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.316598 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.285641 15973 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:16:59.352401 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.391358 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.392033 11471 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:59.392623 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.356479 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.393450 11547 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.364229 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.376032 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.311549 15974 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:59.370759 11473 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:59.372334 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.377902 16004 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:59.310215 16020 consensus_queue.cc:1059] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:16:59.379907 15840 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:59.381888 16028 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.384394 15991 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.282871 16070 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:16:59.392832 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.363497 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.432611 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.432983 11471 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:59.349383 16015 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.419545 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.419847 11365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.433907 11468 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.425150 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.435325 11472 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:59.408211 15974 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.430737 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.396346 11394 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:16:59.440435 11473 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.397334 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.426652 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.441643 11469 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:59.435385 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.443543 11477 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:16:59.419493 16040 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.428392 15840 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:16:59.457875 11471 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:59.459690 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.461283 16006 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:16:59.461875 16006 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.464124 16006 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:16:59.465713 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:16:59.467624 15840 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.468653 15840 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.477730 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 2073) took 1234 ms. Trace:
I20260709 13:16:59.478155 11399 rpcz_store.cc:276] 0709 13:16:58.243373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.295701 (+ 52328us) service_pool.cc:224] Handling call
0709 13:16:59.477515 (+1181814us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54003a7ad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:59.477647 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847764864}
I20260709 13:16:59.480988 16020 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:59.481462 16020 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:16:59.481837 16020 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:59.484077 12118 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:59.484340 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.442009 16015 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.487283 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.488350 15955 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.489140 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.488832 15955 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.489601 11546 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:59.491039 15955 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.492215 11472 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:16:59.488781 11477 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:59.498047 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.499840 11471 raft_consensus.cc:2410] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:16:59.502933 11471 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:59.501508 11547 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:59.504097 11549 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:59.508122 11473 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:59.508172 11469 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:16:59.512259 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.513993 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.516505 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2515) took 1216 ms. Trace:
I20260709 13:16:59.516887 11543 rpcz_store.cc:276] 0709 13:16:58.288586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.288850 (+   264us) service_pool.cc:224] Handling call
0709 13:16:59.504543 (+1215693us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54003ea3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:59.504667 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702274432}
I20260709 13:16:59.488891 16015 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.531494 11549 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:59.530557 16026 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.532838 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.532943 16026 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:16:59.538643 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 118878us
W20260709 13:16:59.529646 16053 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:16:59.540079 11477 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:59.542821 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:59.540526 11468 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:16:59.548559 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:59.529011 15692 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.511828 15336 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:16:59.504851 16040 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.557360 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.538594 16026 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.563098 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:59.551208 16003 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:59.577593 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.578583 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.578847 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.543376 16046 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:16:59.579735 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.580096 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.580281 11470 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:16:59.543514 15765 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:16:59.580855 12118 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:56142
I20260709 13:16:59.581156 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.581282 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.581424 12118 raft_consensus.cc:606] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.581944 11468 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:16:59.582155 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.579066 16040 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.582911 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.584443 11477 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:16:59.584942 11468 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:59.568228 15703 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:59.588238 16020 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.589628 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.590502 16020 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.558622 16076 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:59.591140 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.591226 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.591733 16076 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.592162 16076 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.592164 11477 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:16:59.596289 16003 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.597329 16003 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.560740 15692 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.600934 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.601142 15692 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.601811 11402 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:59.605724 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.609781 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.613137 16070 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:59.613538 16070 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.614274 16070 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.615453 16070 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.615967 16076 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.621762 15703 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:16:59.622689 15703 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:16:59.620219 16076 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.632294 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.633530 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.634240 11548 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 7: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:59.638320 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.644186 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.634961 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.674878 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.673457 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:59.681722 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43483.7us
I20260709 13:16:59.683718 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.684793 11402 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:16:59.644799 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.645432 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.646575 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.646785 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.693527 11470 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:16:59.647909 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.697671 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2518) took 1325 ms. Trace:
I20260709 13:16:59.698073 11542 rpcz_store.cc:276] 0709 13:16:58.372095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.809519 (+437424us) service_pool.cc:224] Handling call
0709 13:16:58.810838 (+  1319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.079501 (+268663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.697440 (+617939us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540041d2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:59.697611 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507351936}
I20260709 13:16:59.699450 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.700564 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.701748 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.705755 15906 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:16:59.691008 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.707497 15099 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:59.707973 15099 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.708843 15099 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.710415 15099 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:59.714011 15776 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.717351 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.717957 11470 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:16:59.720443 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.720933 11470 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.721613 11468 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:59.724614 16004 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:59.725037 16004 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.725926 16004 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.641242 12123 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
W20260709 13:16:59.732153 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.733225 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:16:59.735796 15593 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.736250 15593 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.737463 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.738301 15593 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.738585 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.739391 11546 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:16:59.740152 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.741513 11546 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:59.706255 15906 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.748834 15906 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.750017 11546 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:59.756711 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:16:59.758808 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.759572 11471 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:59.761515 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.764799 16004 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:59.770267 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.772636 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:16:59.773324 11476 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.776844 12123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55066 (request call id 843) took 1082 ms. Trace:
I20260709 13:16:59.778434 11475 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:59.750149 15906 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:59.702695 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.799856 15593 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.802129 15593 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.805388 15593 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.800768 11542 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:16:59.797508 11470 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:16:59.810077 15776 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:59.812263 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.777374 12123 rpcz_store.cc:276] 0709 13:16:58.694025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.694141 (+   116us) service_pool.cc:224] Handling call
0709 13:16:59.776613 (+1082472us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400745254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:59.776748 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099192704}
I20260709 13:16:59.831835 11476 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
W20260709 13:16:59.833871 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 141353us
I20260709 13:16:59.828545 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:16:59.836673 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.837096 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2523) took 1239 ms. Trace:
I20260709 13:16:59.837450 11551 rpcz_store.cc:276] 0709 13:16:58.597905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.615699 (+ 17794us) service_pool.cc:224] Handling call
0709 13:16:59.836947 (+1221248us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400405954. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:16:59.837043 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871122688}
I20260709 13:16:59.837785 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:59.839838 11394 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:59.841135 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 84475.7us
I20260709 13:16:59.844637 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:16:59.848089 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.857681 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.690932 16026 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:59.880698 11401 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:59.883181 15886 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.884920 15886 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.887007 15886 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:16:59.865932 15851 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:16:59.895710 15851 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.897889 15851 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:16:59.876243 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:16:59.853338 16006 raft_consensus.cc:2764] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:16:59.855720 15998 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:59.689150 16001 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.869518 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 179822us
W20260709 13:16:59.869690 16068 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.530s	user 0.005s	sys 0.000s
I20260709 13:16:59.853026 16040 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:59.859130 16015 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:59.904771 11401 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:59.850208 15776 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:59.907099 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.908595 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.909749 15894 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:16:59.903422 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45863.7us
I20260709 13:16:59.918684 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:16:59.826277 15593 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.910760 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 59503.1us
I20260709 13:16:59.825664 11471 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 8.
I20260709 13:16:59.921038 11365 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.866539 16043 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:16:59.925343 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.904135 15851 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:59.931217 12115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 846) took 1128 ms. Trace:
I20260709 13:16:59.935436 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.938848 15099 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:59.939474 15099 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.940009 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.940322 15099 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.930428 16122 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:59.644784 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.950628 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:16:59.932679 12115 rpcz_store.cc:276] 0709 13:16:58.802704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.802881 (+   177us) service_pool.cc:224] Handling call
0709 13:16:58.815438 (+ 12557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:58.989338 (+173900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.989365 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:58.989915 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.124353 (+134438us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:59.124399 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.922623 (+798224us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b5400720454. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:16:59.922703 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.922765 (+    62us) raft_consensus.cc:1685] finished
0709 13:16:59.922785 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.931137 (+  8352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8305,"raft.queue_time_us":9250,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":62,"spinlock_wait_cycles":109674368,"thread_start_us":161471,"threads_started":2,"wal-append.queue_time_us":133701}
I20260709 13:16:59.931743 15894 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.964653 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:16:59.968037 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.916050 16001 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.948679 15955 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:59.969276 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:16:59.972241 16118 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
I20260709 13:16:59.974063 15080 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:59.985587 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:16:59.974900 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:16:59.987914 15080 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:59.988564 15080 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:16:59.997786 11544 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:59.941787 15915 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:16:59.960399 16122 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:00.013458 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:16:59.966058 15638 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:59.967437 15840 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:00.004563 15080 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:16:59.936986 15593 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:16:59.998175 16075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.072s	sys 0.158s
W20260709 13:17:00.012322 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 367567us
I20260709 13:16:59.955927 15416 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:16:59.976380 15955 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.976804 16119 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:00.041101 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:00.050786 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 55177us
I20260709 13:17:00.051823 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:00.057737 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:00.062683 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:00.039935 16075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.072s	sys 0.159s
I20260709 13:17:00.006687 16026 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:17:00.082756 15593 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:00.086128 15593 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:00.076318 16119 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:00.087431 16040 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:00.089214 15593 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:00.090705 15998 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:00.091212 15998 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:00.092190 15998 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.093731 15998 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:00.100283 16001 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:00.101097 16001 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:00.055204 15955 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:00.063454 16006 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:00.069727 15886 raft_consensus.cc:2764] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:17:00.130551 16125 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:17:00.087901 16040 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:00.078389 15751 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 13:17:00.158064 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 106358us
I20260709 13:17:00.158942 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:00.186046 15336 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:00.194972 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 276347us
I20260709 13:17:00.182698 15703 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:00.200485 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:00.219637 15336 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:00.220223 15336 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:00.220290 16072 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:00.224673 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:00.227656 11476 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:00.231016 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:00.239631 11394 raft_consensus.cc:1275] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:00.250363 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:00.190254 15807 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:17:00.256145 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.259423 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:00.263324 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.256219 15851 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:00.135177 15894 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.228930 16043 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:00.220988 16072 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.255192 15703 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:00.273952 16006 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:00.277537 15886 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:00.278680 15807 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:00.087893 16119 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.133847 16122 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:00.298206 15974 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.851s	user 0.011s	sys 0.000s
I20260709 13:17:00.283620 16015 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:00.280413 16001 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:00.280444 15851 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:00.312163 15593 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:00.340572 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 321501us
I20260709 13:17:00.278436 15776 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 13:17:00.324810 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 334361us
I20260709 13:17:00.285485 15638 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:00.262032 16040 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.368865 11471 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:00.326223 15336 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.323858 16006 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.342937 15593 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:00.325784 15886 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.383736 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:00.388236 11469 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:00.366072 15703 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.392477 15886 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:00.334403 15974 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.402458 11475 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:17:00.406234 16133 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:17:00.344164 16001 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.417186 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:00.394156 16040 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:00.417646 11401 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
W20260709 13:17:00.412811 16070 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.797s	user 0.012s	sys 0.001s
W20260709 13:17:00.417054 16126 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:17:00.417225 16134 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:17:00.418025 16127 log.cc:927] Time spent T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 13:17:00.446444 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 51588.2us
I20260709 13:17:00.431771 15840 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:00.372471 15807 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.484079 11397 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:00.470098 16072 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:00.491726 15973 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:00.494448 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:00.497287 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 434680us
I20260709 13:17:00.494460 15973 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.440230 15638 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.498656 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:00.633939 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99528.7us
I20260709 13:17:00.634696 11367 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:00.522976 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:00.526245 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:17:00.598327 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.621390 15080 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 13:17:00.631882 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 372584us
I20260709 13:17:00.641625 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:00.648754 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 150190us
I20260709 13:17:00.684872 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:00.687335 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:00.688911 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 55686.8us
W20260709 13:17:00.689870 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 48325.2us
I20260709 13:17:00.707140 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:00.649128 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:00.695770 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.585302 15991 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:17:00.763134 15099 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:00.605067 16138 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:17:00.517621 15973 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:00.609541 16150 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:00.790696 11474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.016s	sys 0.033s
W20260709 13:17:00.791230 11474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.017s	sys 0.033s
I20260709 13:17:00.648102 15886 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:00.692196 16015 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:00.514748 15840 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:00.633155 16139 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
I20260709 13:17:00.688284 16142 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:00.807655 11300 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 143) took 21085 ms (client timeout 15000 ms). Trace:
I20260709 13:17:00.769951 15915 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:00.809695 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.502951 15955 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.758481 15630 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:17:00.514951 15974 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:00.516696 16006 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:00.528635 16137 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:00.847301 12123 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:00.848976 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:00.607860 16147 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.183s
I20260709 13:17:00.774551 15751 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:00.692091 16043 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 13:17:00.866532 15851 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:00.832914 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 252063us
I20260709 13:17:00.596868 15593 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.769295 15705 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:17:00.769626 16141 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:17:00.871011 16148 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.897351 16139 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:00.805644 16150 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:00.861382 15973 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:17:00.860450 15840 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:00.899211 15886 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:00.916054 16043 consensus_queue.cc:1243] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:17:00.981148 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 274078us
I20260709 13:17:01.019856 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:00.978955 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 217711us
I20260709 13:17:00.899605 15915 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:00.995126 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.998479 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:17:01.001374 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 314129us
I20260709 13:17:01.022189 11475 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.022754 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:00.911731 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:00.906318 15751 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.033583 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.038388 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:00.988821 16139 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:01.036979 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 125373us
I20260709 13:17:01.044020 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:00.903079 16137 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.001694 15886 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:01.050802 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.030840 15851 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:01.042258 11514 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:01.033183 16040 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:00.989264 16148 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.041654 16006 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:01.079639 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:01.042681 16164 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:01.057629 15886 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.085402 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:00.852339 11300 rpcz_store.cc:269] 0709 13:16:39.705593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.803621 (+ 98028us) service_pool.cc:224] Handling call
0709 13:17:00.791415 (+20987794us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:57.754344 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:57.840184 (+ 85840us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:57.840249 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:16:57.840262 (+    13us) tablet.cc:662] Decoding operations
0709 13:16:57.843816 (+  3554us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:57.843919 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:57.843956 (+    37us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:16:57.845699 (+  1743us) tablet.cc:701] Row locks acquired
0709 13:16:57.845755 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:16:57.845994 (+   239us) write_op.cc:270] Start()
0709 13:16:57.846160 (+   166us) write_op.cc:276] Timestamp: P: 1783603017845893 usec, L: 0
0709 13:16:57.846204 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:16:57.872749 (+ 26545us) log.cc:844] Serialized 6632 byte log entry
0709 13:16:58.112625 (+239876us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400018720. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da988edbe 00007f8da9899930 00007f8da98998d8 00007f8da9899858 00007f8da9899813 00007f8da9898a9c 00007f8daf784626 00007f8daf77edd8 00007f8da98d57cb 00007f8da98d82c0
0709 13:16:58.346881 (+234256us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18001b99a0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8da9896190 00007f8da988fae7 00007f8da988edd9 00007f8da9899930
0709 13:16:58.657610 (+310729us) op_driver.cc:464] REPLICATION: finished
0709 13:16:58.792411 (+134801us) write_op.cc:301] APPLY: starting
0709 13:16:58.809192 (+ 16781us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:59.020140 (+210948us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991632a 00007f8da989bf14 00007f8da989bec3 00007f8da989be38 00007f8da989bdf3 00007f8da989b0dc 00007f8da98c4e16 00007f8da98accbb 00007f8da989739d 00007f8da989ad75
0709 13:16:59.158068 (+137928us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:59.158114 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:59.689774 (+531660us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:59.893922 (+204148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:16:59.893987 (+    65us) write_op.cc:312] APPLY: finished
0709 13:16:59.894442 (+   455us) log.cc:844] Serialized 156 byte log entry
0709 13:17:00.213802 (+319360us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.253680 (+ 39878us) write_op.cc:454] Released schema lock
0709 13:17:00.253778 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:00.283433 (+ 29655us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.284827 (+  1394us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3089912,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":78297,"apply.run_cpu_time_us":52629,"apply.run_wall_time_us":1498709,"cfile_cache_hit":66,"cfile_cache_hit_bytes":69003,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":17126,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":10,"num_ops":17,"prepare.queue_time_us":734278,"prepare.run_cpu_time_us":21450,"prepare.run_wall_time_us":392725,"raft.queue_time_us":495450,"raft.run_cpu_time_us":5152,"raft.run_wall_time_us":409884,"replication_time_us":781586,"spinlock_wait_cycles":41906176,"thread_start_us":1266262,"threads_started":5,"wal-append.queue_time_us":629347,"wal-append.run_cpu_time_us":27476,"wal-append.run_wall_time_us":1554149}]]}
I20260709 13:17:01.093966 12123 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:01.097052 16003 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:17:01.099953 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.057850 15851 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:01.114137 11475 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:01.115556 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.157601 16174 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:01.123025 16043 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:01.122589 16006 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.166738 11475 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:01.125383 16015 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.171370 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:01.162844 15955 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:01.155392 16144 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:17:01.130330 15718 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.081166 16040 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.160607 16181 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:01.190830 16043 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.178913 11397 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:01.162185 15973 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.080826 16173 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:01.174098 16156 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 13:17:01.195212 16159 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 13:17:01.194562 16160 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:17:01.197607 16040 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.197870 16186 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.106401 15915 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.198307 16150 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:01.207494 16167 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:17:01.191083 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 111538us
W20260709 13:17:01.238957 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 67765.2us
I20260709 13:17:01.117168 16139 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.207104 15973 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:01.194024 16162 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:01.274705 11476 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:01.269447 16137 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.199855 16043 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:01.194936 16163 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:17:01.286219 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:01.299970 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.263115 15751 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:01.256660 16158 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
W20260709 13:17:01.301270 11367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.023s
I20260709 13:17:01.198686 16181 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:01.285456 15718 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.309434 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:01.313762 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 198316us
I20260709 13:17:01.309513 11472 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:17:01.312456 11367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.000s	sys 0.024s
I20260709 13:17:01.314616 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:01.315653 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 530352us
I20260709 13:17:01.317556 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:01.317715 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:01.177232 15851 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.226342 16015 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.328153 11469 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:01.330847 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:01.308753 15840 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.332293 11469 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:01.302049 16171 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:17:01.318692 15718 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.305365 15768 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:17:01.335283 16185 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:01.345196 15851 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:01.345690 16185 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.346259 16185 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.335343 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:01.316370 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.328334 15705 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:01.351576 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.327023 15751 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:17:01.364660 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48367.9us
I20260709 13:17:01.365557 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.371322 11442 leader_election.cc:400] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:17:01.372468 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.347414 16173 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:01.339500 16148 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:01.345001 15973 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.374399 16171 consensus_queue.cc:1243] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:01.363300 16141 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.395617 16141 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.398030 16141 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:01.377460 16184 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:17:01.359273 16137 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:01.359460 16069 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:17:01.363322 16170 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:17:01.377143 15336 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:01.407876 16001 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.375748 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.370195 16176 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:01.410141 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.411299 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.374838 16194 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.379357 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:01.414790 11397 raft_consensus.cc:606] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.419720 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.422891 11547 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:01.431849 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:01.433696 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:01.434484 11549 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.364018 15776 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:01.380546 16181 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:01.439085 16173 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.448239 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.438678 16192 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:01.467018 16173 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.467480 11472 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:01.377041 16182 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.459376 15894 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:01.468823 15894 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:01.469348 16173 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:01.441066 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.469671 15894 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.464098 15840 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:01.473364 16192 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.474269 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.475136 11546 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:01.475070 15776 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:01.476105 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.476100 15776 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.476917 11550 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:01.477342 15776 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.478332 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.478927 11550 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 4.
I20260709 13:17:01.479275 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.479858 11544 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:01.480536 16176 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.481952 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2544) took 1096 ms. Trace:
I20260709 13:17:01.485646 16180 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.486209 16180 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.487425 16181 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:01.487869 16181 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:01.488476 16180 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:01.489219 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.489959 15998 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.490445 15998 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.491361 16137 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.491962 16137 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.492679 15998 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.493021 15974 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:01.493427 15974 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:01.495190 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:01.496863 16190 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:01.497440 16122 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:01.498826 16001 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.482096 16069 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.342880 15886 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.470723 15894 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.500121 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:01.504330 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.482501 11473 rpcz_store.cc:276] 0709 13:17:00.385121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.396259 (+ 11138us) service_pool.cc:224] Handling call
0709 13:17:00.411319 (+ 15060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:00.442535 (+ 31216us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:00.649088 (+206553us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.000169 (+351081us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.000288 (+   119us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:01.000692 (+   404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.480848 (+480156us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180049b760. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419
0709 13:17:01.481685 (+   837us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5400062a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:01.481742 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.481886 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22559,"prepare.queue_time_us":454693,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":5939,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":14060,"spinlock_wait_cycles":77535744,"thread_start_us":530706,"threads_started":2,"wal-append.queue_time_us":622157}
I20260709 13:17:01.494277 15974 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:01.556540 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56485.4us
W20260709 13:17:01.378046 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61564us
W20260709 13:17:01.463588 16169 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:17:01.555464 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.550267 11472 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:01.455142 16188 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:01.559424 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.572286 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.572700 15705 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:01.573094 15705 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.573442 15705 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.573360 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:01.574573 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.576576 15851 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.556288 15998 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.577066 15851 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.577732 15998 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.562222 16004 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.580068 16004 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.582543 16004 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:01.502709 16190 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:01.560288 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.502769 15886 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.498962 15593 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.563575 11469 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:17:01.593432 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.593521 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.561960 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.564829 16137 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.563040 16192 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.562868 16142 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.566185 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:01.596701 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:01.596866 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:01.597268 11397 raft_consensus.cc:606] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:01.597594 11397 raft_consensus.cc:614] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:01.501292 15638 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:17:01.584923 11547 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:01.500851 15630 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:17:01.376428 15718 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.587491 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.561269 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.589807 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.594832 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:01.613627 11477 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:17:01.563237 16001 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.590855 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:01.566648 16015 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:01.622727 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.501549 16020 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:17:01.600587 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:01.540333 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.597923 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:01.501441 15973 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:17:01.499711 16069 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.610682 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.632655 11548 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:01.594941 16140 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.646163 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.481014 16176 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.632968 16137 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.603598 16188 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:01.677405 11473 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:01.603368 16194 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:01.429467 16043 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.644812 16015 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.687553 16015 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.687935 15840 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:01.603826 16192 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.635073 15718 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.692719 15718 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.693632 16015 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:01.693795 16192 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:01.678896 16188 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.694475 16188 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.689182 11475 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:17:01.682461 16194 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.697715 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:01.661839 11515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.644812 16006 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:01.700062 11551 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:01.703924 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:01.675185 16040 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.652048 15751 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.706799 15751 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.708443 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.709132 11551 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:01.651857 16173 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.694813 15718 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.682822 15998 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:01.716660 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 413431us
I20260709 13:17:01.712397 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55066 (request call id 858) took 1090 ms. Trace:
I20260709 13:17:01.652454 15851 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:01.699048 16194 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.681867 11549 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:01.719907 11543 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:01.724365 11548 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:01.650036 16001 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.682705 16137 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.723158 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.613619 16142 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:01.722656 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.731055 11473 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:01.704942 16015 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.731057 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.705406 16006 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:01.747740 16006 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:01.746719 11397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:01.735260 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.752372 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.756654 15336 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:01.761185 16180 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:01.767596 15840 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.768306 15840 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.773238 15630 raft_consensus.cc:993] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:01.773953 15630 raft_consensus.cc:1081] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
W20260709 13:17:01.719934 16149 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.681s	user 0.004s	sys 0.000s
I20260709 13:17:01.777101 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 809) took 1590 ms. Trace:
I20260709 13:17:01.767686 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2543) took 1385 ms. Trace:
I20260709 13:17:01.798358 11468 rpcz_store.cc:276] 0709 13:17:00.382619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.382753 (+   134us) service_pool.cc:224] Handling call
0709 13:17:00.383090 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:00.393638 (+ 10548us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:00.608013 (+214375us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.321263 (+713250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.321332 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:01.321393 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.767447 (+446054us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b540042a9d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:01.767492 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.767620 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14472,"prepare.queue_time_us":475400,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":103685,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":73503,"spinlock_wait_cycles":23715328,"thread_start_us":886790,"threads_started":3,"wal-append.queue_time_us":572104}
I20260709 13:17:01.801999 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.831667 11475 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:17:01.832322 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:01.838315 11468 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:01.777571 11474 rpcz_store.cc:276] 0709 13:17:00.187708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.188277 (+   569us) service_pool.cc:224] Handling call
0709 13:17:00.188640 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:00.189062 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:00.189099 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:00.189638 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:17:00.791710 (+602072us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:00.791784 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.791883 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.694707 (+902824us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:01.776711 (+ 82004us) raft_consensus.cc:1685] finished
0709 13:17:01.776739 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.777022 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":925,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10926,"spinlock_wait_cycles":2602752,"thread_start_us":528894,"threads_started":1,"wal-append.queue_time_us":697450}
I20260709 13:17:01.872721 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:01.873939 11547 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:01.894021 11469 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:17:01.653041 16140 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.727348 16192 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:01.705276 15906 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:17:01.731377 15718 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:01.902105 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 69870.8us
I20260709 13:17:01.906926 11474 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:17:01.922567 11549 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:01.725430 15765 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.174s
I20260709 13:17:01.722834 16173 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:01.933176 16202 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:17:01.944425 16001 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:01.718668 16040 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.704993 16190 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.908188 15807 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:01.954759 11547 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:01.971467 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 360889us
I20260709 13:17:01.745749 16015 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:01.894447 16181 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:01.987362 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.987546 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:01.993796 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:02.000782 11549 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:02.004666 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:02.006415 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:02.006640 15099 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.007261 15630 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:02.007833 15630 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.008160 15630 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:02.010677 11547 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:02.012374 15099 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.014894 15336 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:01.974540 16182 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:01.969112 15593 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.928002 15718 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.019407 11545 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 7.
I20260709 13:17:02.020511 15099 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:01.922175 16192 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:01.951236 15705 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:02.020870 16185 consensus_meta.cc:287] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.674s	user 0.004s	sys 0.011s
I20260709 13:17:02.018599 15336 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:02.054672 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 145912us
W20260709 13:17:02.040035 15776 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.003s
I20260709 13:17:02.021865 15703 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.063619 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.064900 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.069593 12123 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:02.069543 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.070174 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.072896 11397 raft_consensus.cc:606] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:02.054176 15336 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.077668 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:02.059203 16069 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.029846 15593 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.087992 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.088217 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.093325 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.056739 16185 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:02.086236 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 333499us
I20260709 13:17:02.061164 16199 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 13:17:02.089849 16201 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
W20260709 13:17:02.108915 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.723338 12113 rpcz_store.cc:276] 0709 13:17:00.621665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.621788 (+   123us) service_pool.cc:224] Handling call
0709 13:17:00.684052 (+ 62264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:00.684266 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:17:01.254305 (+570039us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.712037 (+457732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.712094 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:01.712137 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.712206 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.712339 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":586863,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":90,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2788,"thread_start_us":944788,"threads_started":2}
I20260709 13:17:02.110865 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.113492 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:02.095674 11542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.015s	sys 0.024s
I20260709 13:17:02.103354 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.122597 11400 raft_consensus.cc:2393] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:02.089730 15751 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.124655 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:02.125295 11542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.016s	sys 0.024s
I20260709 13:17:02.130636 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.090502 15765 consensus_queue.cc:1048] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:17:02.095347 16176 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.074297 15703 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.141913 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.143389 11550 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:02.142614 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:02.142040 11366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.011s	sys 0.030s
W20260709 13:17:02.144863 11366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.011s	sys 0.030s
I20260709 13:17:02.102792 16142 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:02.147056 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.123641 16137 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.137424 16140 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:02.156705 16208 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:17:02.140548 16184 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:02.201257 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:02.211788 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.141490 16220 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.146416 16003 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:02.202534 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.207484 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:02.222200 11394 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
W20260709 13:17:02.219830 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 212558us
I20260709 13:17:02.142033 16069 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:17:02.219586 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 139742us
W20260709 13:17:02.197600 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 90190.7us
I20260709 13:17:02.220806 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.158861 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.205330 11297 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.207501 16124 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:02.209872 16173 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.247071 11297 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.249800 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:02.266548 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.267568 12114 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:02.231617 16003 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:02.240741 16207 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:02.340020 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.339031 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.338725 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:02.341452 11514 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.030s	sys 0.060s
W20260709 13:17:02.341868 11514 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.030s	sys 0.060s
W20260709 13:17:02.342204 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 667236us
I20260709 13:17:02.241225 16181 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.345702 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:02.346395 16205 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:17:02.354502 11514 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:02.343858 16196 log.cc:927] Time spent T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:17:02.401208 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.402137 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.362339 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.362457 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:02.401219 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.361739 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.403304 12113 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:02.340015 16003 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:02.338753 15705 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.338816 16015 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:02.360231 11297 catalog_manager.cc:5697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 8, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 8 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.343254 16207 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.225199 16184 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.346765 15776 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:02.362193 16173 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:02.409989 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47624us
I20260709 13:17:02.241016 16211 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 13:17:02.416688 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.420626 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:02.422243 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.423341 16015 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:02.359335 16221 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:17:02.430197 15336 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.417018 16207 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:02.432476 15336 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:02.435600 16216 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:17:02.412837 16231 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.436575 15703 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.414232 15751 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:02.440986 11297 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:02.442458 16006 consensus_meta.cc:287] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.694s	user 0.007s	sys 0.001s
I20260709 13:17:02.442657 11297 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 8, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 8 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.410205 16137 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:02.436384 16231 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.444296 11297 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 10 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.446053 11297 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.447940 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:02.449083 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:02.450362 15751 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.410362 16124 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:02.451021 15751 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:02.451789 16124 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.452239 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:02.453275 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.454449 15751 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:02.454905 15751 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.455241 15751 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:02.459280 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2555) took 1079 ms. Trace:
W20260709 13:17:02.460638 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.459661 11542 rpcz_store.cc:276] 0709 13:17:01.380263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.381182 (+   919us) service_pool.cc:224] Handling call
0709 13:17:01.381482 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.381903 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.381942 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.382488 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:17:02.136727 (+754239us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:02.136765 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.136825 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.457920 (+321095us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:02.459005 (+  1085us) raft_consensus.cc:1685] finished
0709 13:17:02.459024 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.459224 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9271,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"spinlock_wait_cycles":83285376,"thread_start_us":632943,"threads_started":1,"wal-append.queue_time_us":833579}
W20260709 13:17:02.463135 16213 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.246s	user 0.002s	sys 0.000s
I20260709 13:17:02.464018 15630 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.446769 16231 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.469107 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:02.469852 11474 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.472294 16207 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.475457 15630 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.481503 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.469192 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.484683 16185 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.485019 11400 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:17:02.418781 15593 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:02.476892 16229 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:02.479894 16192 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.478555 16184 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.493454 11474 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:02.442888 16006 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.494518 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:02.474916 16188 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.780s	user 0.000s	sys 0.010s
I20260709 13:17:02.495682 16006 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.418824 15776 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.496899 15776 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:02.495325 16188 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.499607 11470 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:02.500689 16188 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.501431 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:02.507555 11470 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 16: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:02.508414 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:02.509038 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 98184.1us
I20260709 13:17:02.514143 11470 raft_consensus.cc:2410] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:17:02.516075 16230 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.516654 16230 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:02.520032 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 67869.3us
I20260709 13:17:02.520728 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:02.523211 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.525641 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.518740 16230 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:02.531944 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.532939 11394 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:02.533607 11474 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:02.541414 11400 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:02.592181 11549 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:02.592859 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:02.537906 16194 consensus_meta.cc:287] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.804s	user 0.006s	sys 0.005s
I20260709 13:17:02.597824 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:02.598518 11544 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:17:02.537704 16184 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.647265 11297 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.599041 15593 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.600924 16173 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.649744 15703 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.664899 15776 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.603983 16194 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.666608 16173 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.661237 11400 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:02.668893 11297 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 10 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.693394 16139 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:02.698320 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 456918us
I20260709 13:17:02.599499 16003 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.707856 11472 raft_consensus.cc:1240] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:02.700644 11297 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 10 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.699674 12116 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 15: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:02.699369 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:02.709578 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 480958us
I20260709 13:17:02.731397 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:02.654050 16220 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:17:02.741003 11297 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 3 to 5, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 5 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.742735 11297 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 5, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 5 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.745504 11297 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 11, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 11 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:02.673100 15703 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.753880 15703 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:02.671511 15593 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.756462 15593 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:02.759379 11550 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 2142) took 1175 ms (client timeout 1538 ms). Trace:
W20260709 13:17:02.759814 11550 rpcz_store.cc:269] 0709 13:17:01.584061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.590563 (+  6502us) service_pool.cc:224] Handling call
0709 13:17:02.759168 (+1168605us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54001a77d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:02.759316 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202383616}
I20260709 13:17:02.761008 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:02.763492 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2579) took 1021 ms. Trace:
W20260709 13:17:02.764151 11284 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34fe1737ebb840fb9746980f649cc0a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:02.766537 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 638) took 1102 ms. Trace:
I20260709 13:17:02.768122 11549 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:02.768177 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:02.766952 11551 rpcz_store.cc:276] 0709 13:17:01.664206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.717090 (+ 52884us) service_pool.cc:224] Handling call
0709 13:17:01.717385 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.899920 (+182535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.899953 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.900001 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:01.900032 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.766292 (+866260us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002067d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:02.766350 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.766481 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419219200}
I20260709 13:17:02.770391 11550 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:02.763577 16194 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:02.764027 11548 rpcz_store.cc:276] 0709 13:17:01.742328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.022613 (+280285us) service_pool.cc:224] Handling call
0709 13:17:02.053419 (+ 30806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.057152 (+  3733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.057198 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:02.057252 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:02.057316 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.763285 (+705969us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b54001a7cd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:02.763335 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.763440 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8032512}
I20260709 13:17:02.779551 11475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1765) took 1093 ms. Trace:
I20260709 13:17:02.770707 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:02.781267 11551 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.780124 11475 rpcz_store.cc:276] 0709 13:17:01.673941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.697359 (+ 23418us) service_pool.cc:224] Handling call
0709 13:17:02.767283 (+1069924us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540005a854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:02.767405 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306112256}
I20260709 13:17:02.784761 11297 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 11, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 11 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:02.796723 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 192483us
I20260709 13:17:02.740571 16015 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:02.810194 11297 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.817109 11551 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:02.822245 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:02.824288 11551 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.773438 16184 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:02.833024 11297 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.840790 11297 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:02.836206 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:02.845618 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:02.846194 11548 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:02.809891 16015 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.807201 16234 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.834852 16184 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.862618 11551 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:17:02.864164 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 132850us
I20260709 13:17:02.860765 11297 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.856853 16015 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.832783 15751 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.880072 11545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43430 (request call id 2582) took 1133 ms. Trace:
I20260709 13:17:02.886390 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:02.883764 11545 rpcz_store.cc:276] 0709 13:17:01.746408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.752042 (+  5634us) service_pool.cc:224] Handling call
0709 13:17:02.879907 (+1127865us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002076d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:02.880011 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612330240}
I20260709 13:17:02.889415 15974 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:02.892516 15768 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:02.892925 15768 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:02.893743 15768 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.894764 15768 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:02.895205 16239 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:17:02.905109 16137 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.905794 16137 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:02.912833 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.868350 16184 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.877071 16215 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:02.931258 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:02.874956 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:02.934935 11551 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.935420 11551 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.877357 15593 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:17:02.917172 16175 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:02.863508 16234 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.933764 16215 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:02.980921 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64665.4us
I20260709 13:17:02.984560 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:02.964473 11551 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:02.959383 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:02.933102 15776 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:02.990613 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.991202 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.991792 11394 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:02.991825 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.988024 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:02.992738 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:03.007735 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:03.008663 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:03.009543 11395 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:02.954888 16250 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.013459 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:02.959227 16173 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.014655 11398 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:02.988746 16215 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:02.990391 16003 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.019440 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.067669 11545 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:17:03.048025 11393 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:03.030905 11398 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:03.075538 11476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2581) took 1099 ms. Trace:
I20260709 13:17:03.009580 16227 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.050616 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2573) took 1490 ms. Trace:
I20260709 13:17:03.075992 11476 rpcz_store.cc:276] 0709 13:17:01.976546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.036515 (+ 59969us) service_pool.cc:224] Handling call
0709 13:17:02.036922 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:02.037393 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.037433 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:02.038087 (+   654us) log.cc:844] Serialized 52 byte log entry
0709 13:17:02.206788 (+168701us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:02.223665 (+ 16877us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.075217 (+851552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.075289 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.075328 (+    39us) raft_consensus.cc:1685] finished
0709 13:17:03.075345 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.075481 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1837160,"prepare.queue_time_us":16162,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":5891,"raft.queue_time_us":117,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5488,"thread_start_us":151453,"threads_started":1,"wal-append.queue_time_us":303234}
I20260709 13:17:03.084928 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.085707 11546 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:17:03.087196 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:03.089491 16263 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:03.090026 16263 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:02.980302 16175 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.094950 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.094830 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.097697 11546 raft_consensus.cc:1240] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:03.099442 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110658us
I20260709 13:17:03.100306 11546 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:03.076781 11477 rpcz_store.cc:276] 0709 13:17:01.560403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.632234 (+ 71831us) service_pool.cc:224] Handling call
0709 13:17:01.632494 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.632678 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:02.075594 (+442916us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.050277 (+974683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:03.050332 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:03.050373 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.050441 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.050555 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":992170,"prepare.queue_time_us":49892,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":991042,"thread_start_us":424226,"threads_started":1}
I20260709 13:17:03.117231 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2554) took 1737 ms. Trace:
I20260709 13:17:03.118818 11543 rpcz_store.cc:276] 0709 13:17:01.379923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.380710 (+   787us) service_pool.cc:224] Handling call
0709 13:17:01.381021 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.724369 (+343348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.116957 (+1392588us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b5400269154. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:03.117172 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771945856}
W20260709 13:17:03.129284 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 118069us
W20260709 13:17:03.084393 16249 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 13:17:03.136963 11543 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:17:03.077939 15973 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.140753 15973 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.133200 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:03.032143 16173 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.135525 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.106289 11365 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:17:03.084687 16241 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.134s	user 0.000s	sys 0.003s
I20260709 13:17:03.142935 12085 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:03.138098 11402 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:03.043514 16137 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.036554 16234 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.105198 16006 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.103788 16188 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.117672 16142 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.160760 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:03.162431 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.067014 16250 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.167204 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 66482.6us
I20260709 13:17:03.141973 16173 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:03.170099 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 63899.3us
I20260709 13:17:03.155975 16186 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.172780 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.173823 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.175388 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.176429 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.157730 16192 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.160197 16234 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.179381 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.181869 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.185787 11548 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:03.185918 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.161115 15886 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.163458 15765 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:03.163252 16182 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.213258 11475 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:03.188831 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.214682 11477 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:03.215706 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.216801 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.213344 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.222025 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.223093 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.223735 11475 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:17:03.215047 16190 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:17:03.224893 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:03.226444 16252 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:17:03.177981 16001 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.227581 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2580) took 1251 ms. Trace:
I20260709 13:17:03.227329 16001 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.227948 11473 rpcz_store.cc:276] 0709 13:17:01.976241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.996227 (+ 19986us) service_pool.cc:224] Handling call
0709 13:17:01.996588 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:02.002625 (+  6037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.002694 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:02.003278 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:17:02.751603 (+748325us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b1800158440. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:17:02.871709 (+120106us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:02.871747 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.871811 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.045347 (+173536us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b1800158440. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67a6c0a 00007f8dafee6219 00007f8dad7ed58f 00007f8da682c571 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd
0709 13:17:03.227385 (+182038us) raft_consensus.cc:1685] finished
0709 13:17:03.227406 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.227529 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8146,"raft.queue_time_us":22083,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6077,"spinlock_wait_cycles":50353792,"thread_start_us":119484,"threads_started":1,"wal-append.queue_time_us":934137}
I20260709 13:17:03.229681 16260 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.229817 16001 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.230449 16001 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.230367 16219 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:17:03.231835 16218 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:17:03.233083 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2571) took 1673 ms. Trace:
I20260709 13:17:03.233477 11471 rpcz_store.cc:276] 0709 13:17:01.559815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.601911 (+ 42096us) service_pool.cc:224] Handling call
0709 13:17:01.602179 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.602516 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:01.655774 (+ 53258us) log.cc:844] Serialized 14 byte log entry
0709 13:17:02.424158 (+768384us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b1800332c80. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c3f3 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:17:03.053149 (+628991us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:03.053651 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:17:03.053816 (+   165us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:03.053857 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.053939 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.232797 (+178858us) raft_consensus.cc:1685] finished
0709 13:17:03.232902 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.233025 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":971032,"prepare.queue_time_us":48447,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":956051,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":49029,"spinlock_wait_cycles":50171264,"thread_start_us":426270,"threads_started":1,"wal-append.queue_time_us":674188}
I20260709 13:17:03.177578 16194 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.235870 16194 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.237277 16173 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.238607 16194 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.238899 16250 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.248178 16194 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:03.249823 16243 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:17:03.252193 16003 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.252568 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48200 (request call id 28) took 1506 ms. Trace:
I20260709 13:17:03.252977 11397 rpcz_store.cc:276] 0709 13:17:01.746447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.746596 (+   149us) service_pool.cc:224] Handling call
0709 13:17:03.252412 (+1505816us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400348354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9913853 00007f8dae8e160a 00007f8da7e677d4 00007f8da7e67762 00007f8da7e67691 00007f8da7e6760c 00007f8da7e668c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:03.252486 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745356160}
I20260709 13:17:03.240301 16175 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.241431 16015 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.160847 16006 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.188400 16186 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.188027 15751 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.219323 16182 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.219451 15886 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.244493 15765 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:03.245723 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.214696 16192 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.177929 16181 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.187788 16235 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:03.270236 16259 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.312585 16006 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.271919 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.313650 15718 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.315445 16235 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.316207 16182 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.316432 16235 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.317333 15973 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.318162 16015 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.230198 16260 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.284116 16190 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.319846 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1853) took 1259 ms. Trace:
I20260709 13:17:03.281123 15894 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:03.320436 15894 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.321679 16260 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:03.222548 16139 consensus_meta.cc:287] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:17:03.322407 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:03.277947 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:03.322930 15765 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.325645 11401 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:17:03.326026 11401 pending_rounds.cc:85] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 8
I20260709 13:17:03.326259 11401 pending_rounds.cc:107] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:17:03.326488 11401 raft_consensus.cc:2889] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:03.250509 16184 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.327050 15080 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.327512 15080 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:03.275331 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89423.7us
I20260709 13:17:03.320262 11402 rpcz_store.cc:276] 0709 13:17:02.060477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.060592 (+   115us) service_pool.cc:224] Handling call
0709 13:17:02.060882 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.151639 (+1090757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.319606 (+167967us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400347e54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:03.319791 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476970880}
I20260709 13:17:03.328372 11474 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 7.
I20260709 13:17:03.328701 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.118352 16263 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.305351 15886 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.168467 16142 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.321419 15894 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.295706 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79876.2us
I20260709 13:17:03.252718 16003 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.165808 16188 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.335959 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:03.340056 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.345070 11471 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:03.345079 11367 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.344084 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.347095 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.347848 11471 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:03.348686 15638 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:03.349284 15638 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.349406 16190 consensus_queue.cc:1059] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:03.350908 16190 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:03.350178 15638 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.351316 16190 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.352027 15991 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:03.352190 16190 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.353199 16190 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:03.353928 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 129125us
I20260709 13:17:03.356561 11471 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:03.360841 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.361510 11473 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.362434 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.329414 16181 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.363610 15894 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:03.365166 16181 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.367327 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.359107 15886 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:03.369170 16265 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.369293 15886 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.369810 15886 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:03.369774 16265 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.369187 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:03.375269 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.379068 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.272171 16001 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.358191 16020 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:17:03.382089 16001 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.382413 16040 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.387049 11473 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:17:03.337035 16182 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:03.390434 11475 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:03.395569 11476 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:03.398730 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:03.435845 11549 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:03.438019 16070 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:03.438540 16142 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.436947 16260 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.367897 16006 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.388296 16263 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:03.439409 16260 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.441514 16260 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.445758 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.447852 16192 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.449556 16003 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:03.436023 11515 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.732s (SENT) [suppressed 6 similar messages]
I20260709 13:17:03.456501 11547 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:03.438616 16070 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.454758 11543 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:03.390607 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:03.389379 16181 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.475662 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.463950 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.477965 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.478619 11543 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:17:03.395253 16265 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.479487 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:03.480370 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.482599 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 46604us
I20260709 13:17:03.484838 11543 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:03.435132 16184 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.450640 16001 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.450291 16194 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:03.511400 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 120492us
I20260709 13:17:03.512212 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:03.397048 16141 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:17:03.521446 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.522128 12112 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:03.529738 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.537204 11475 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:17:03.539134 16070 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.539793 16279 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:17:03.542799 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 182752us
I20260709 13:17:03.543248 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.544384 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.531313 11367 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.553184 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.554404 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.556682 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.557588 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.558235 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.562605 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.563715 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:03.513670 16265 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.563932 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.549453 11514 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.565299 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.562858 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.572897 11471 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:03.569396 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.573443 11473 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:03.574224 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:03.532824 16181 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.576014 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:03.492954 16258 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:17:03.533766 16006 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:03.596426 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:03.579859 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48783.9us
I20260709 13:17:03.593706 11514 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:03.545859 16182 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.603540 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.581749 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:03.579766 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.605152 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.581749 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.528785 16270 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:03.609505 11477 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:17:03.520850 16001 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.533862 15973 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:17:03.668186 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.668867 11473 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:03.571702 16142 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:03.565209 16192 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.612979 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.620913 11471 raft_consensus.cc:2410] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:03.580143 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.613368 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:03.673623 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 60337.1us
I20260709 13:17:03.602555 16260 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:03.577365 15886 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.641835 16274 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.637279 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.643352 16261 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:03.658150 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.659636 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.662901 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.529241 16194 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.668232 15973 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:03.678951 15973 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.596966 16003 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:03.664635 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.579501 16184 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:03.669907 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 206028us
I20260709 13:17:03.599911 16006 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:03.677606 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.680477 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.681252 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.682837 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.564042 16277 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.185s
I20260709 13:17:03.583639 16265 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.703423 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.691223 11477 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:17:03.716910 11550 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
W20260709 13:17:03.707036 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 94189.3us
I20260709 13:17:03.720206 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.596819 16181 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.696029 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 115956us
W20260709 13:17:03.712764 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 49956.2us
I20260709 13:17:03.725723 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.728178 11471 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 15: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:03.729219 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.684154 11543 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:03.717037 15751 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.680207 16261 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.719002 16003 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.683872 16263 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:03.683646 16260 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.820122 11473 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:03.815784 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.812598 16263 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.774247 11545 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:03.716279 16006 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.723757 16182 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:03.782321 16264 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:03.721110 16292 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:03.735116 16277 consensus_queue.cc:1243] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:03.723348 16004 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:17:03.821954 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 184724us
I20260709 13:17:03.736029 16188 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.721062 16286 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:03.740324 16175 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.823715 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.717890 16184 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:03.815605 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86809.5us
I20260709 13:17:03.806653 11550 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:03.684491 15973 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.813071 16003 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:03.684583 16274 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.839458 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.856791 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:03.814431 16192 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.843792 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.857664 11545 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:17:03.832361 16119 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.844859 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:03.822829 16260 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:03.828078 16263 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.854897 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:03.844728 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.862099 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.862208 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.863798 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:03.868438 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.868813 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:03.835424 16292 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.851572 16001 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.878353 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:03.851289 16286 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.854036 16184 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:03.877069 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:03.884034 16284 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:17:03.890651 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.891876 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.892709 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:03.872977 16190 consensus_meta.cc:287] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.004s
I20260709 13:17:03.867764 16142 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:03.867605 16182 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.863564 16188 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.836261 16264 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.879467 16292 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:03.873581 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:03.920202 15768 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:03.956410 15768 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.957052 16181 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.958731 15768 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:03.937685 16299 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:17:03.921526 16175 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:03.960764 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 79775.3us
I20260709 13:17:03.868135 16192 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.961778 16288 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:17:03.961989 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:03.963201 16192 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:03.938421 16291 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:17:03.938179 16182 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.963788 16264 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.881099 16286 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:03.964303 15718 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:03.964859 15718 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:03.965260 16294 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:03.965564 16182 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.920495 16142 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:03.965752 16294 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:03.966269 16142 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:03.968259 16265 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:03.940240 16274 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.940896 16275 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:03.870182 16119 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.970201 16275 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:03.972729 16275 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:03.942080 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.875582 15751 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:03.952204 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:03.974283 11473 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:17:03.852312 16004 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:03.879251 16194 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:03.961081 16175 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:03.941259 16293 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:17:03.972466 16119 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:03.878579 15973 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:03.974880 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.045281 11468 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
W20260709 13:17:03.963708 16247 log.cc:927] Time spent T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.031s	user 0.000s	sys 0.004s
W20260709 13:17:04.032814 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57386us
I20260709 13:17:04.041857 16303 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:17:03.966817 16294 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.052567 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.053149 11473 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:03.973917 15751 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.054136 11474 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:03.858892 16261 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.056813 16261 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:04.058333 16207 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.062395 16043 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:04.070847 16276 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.610s	user 0.000s	sys 0.002s
I20260709 13:17:04.075922 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2582) took 2099 ms. Trace:
I20260709 13:17:04.076313 11469 rpcz_store.cc:276] 0709 13:17:01.976843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.084215 (+107372us) service_pool.cc:224] Handling call
0709 13:17:02.084556 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:03.051783 (+967227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:03.051816 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:03.052397 (+   581us) log.cc:844] Serialized 52 byte log entry
0709 13:17:03.272692 (+220295us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:03.736612 (+463920us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.799972 (+ 63360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.800081 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.070217 (+270136us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:04.075612 (+  5395us) raft_consensus.cc:1685] finished
0709 13:17:04.075717 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.075862 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":966791,"prepare.queue_time_us":584443,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":1063912,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":17109,"spinlock_wait_cycles":34051968,"thread_start_us":668274,"threads_started":2,"wal-append.queue_time_us":289404}
I20260709 13:17:04.096701 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.099097 11469 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:04.101126 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:04.099261 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 242919us
I20260709 13:17:04.104561 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.105870 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:04.108619 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:04.109061 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.109975 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.110608 11394 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:17:04.110818 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.111639 11397 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:04.110790 11402 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 17. Current term is 18. Ops: []
I20260709 13:17:04.106917 11476 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:17:04.119083 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:04.049300 15886 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.080570 15751 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:04.129035 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 255494us
I20260709 13:17:04.139997 16175 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:04.132637 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.142549 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:04.145648 11476 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:04.150672 11476 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:03.965689 15718 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.156473 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.156870 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.120234 16188 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:04.107810 16194 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.162356 11546 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:04.164955 11472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 828) took 2417 ms. Trace:
I20260709 13:17:04.164855 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.165422 11472 rpcz_store.cc:276] 0709 13:17:01.738715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.914644 (+175929us) service_pool.cc:224] Handling call
0709 13:17:01.914929 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.728834 (+813905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.156178 (+1427344us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b540038a4d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:04.156380 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804883328}
I20260709 13:17:04.127785 16274 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:04.166898 11442 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.093890 16292 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.141095 16182 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:04.229653 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:04.379415 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 212690us
W20260709 13:17:04.378566 12123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55076 (request call id 644) took 2577 ms (client timeout 2631 ms). Trace:
W20260709 13:17:04.386291 12123 rpcz_store.cc:269] 0709 13:17:01.593664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.595692 (+  2028us) service_pool.cc:224] Handling call
0709 13:17:04.171119 (+2575427us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54004dd854. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:04.171245 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025911040}
I20260709 13:17:04.433069 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.433470 11472 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:04.154381 16192 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:04.392686 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.437158 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.158972 16181 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:04.491194 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 360640us
I20260709 13:17:04.496245 11395 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:04.435030 15886 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:04.448998 16260 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.569s	user 0.005s	sys 0.004s
W20260709 13:17:04.440267 16276 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.282s	user 0.000s	sys 0.000s
I20260709 13:17:04.442401 16192 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.515758 16192 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:04.518364 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:04.495213 16215 consensus_meta.cc:287] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 1.392s	user 0.005s	sys 0.007s
I20260709 13:17:04.475965 15840 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.519228 12123 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:04.519619 15840 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:04.519248 16215 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.175942 16301 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:04.520424 16301 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.520987 16301 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:04.522027 16215 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.518591 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:04.505115 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:04.531032 16192 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.534861 16192 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.379303 16001 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:04.542963 16043 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.543666 16043 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.546693 16043 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:04.492710 16274 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:04.552037 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.496357 16182 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:04.563221 11474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.046s
W20260709 13:17:04.563751 11474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.017s	sys 0.046s
I20260709 13:17:04.553032 16260 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.182267 16188 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.382776 16194 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:04.590339 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 153250us
I20260709 13:17:04.591248 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.592602 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:04.601847 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:04.602932 11397 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:04.573557 16182 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.604236 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.605778 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.564996 16274 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:04.572245 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 67246.4us
I20260709 13:17:04.617091 11514 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:04.619626 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 682) took 3427 ms. Trace:
I20260709 13:17:04.620205 11401 rpcz_store.cc:276] 0709 13:17:01.182769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.182929 (+   160us) service_pool.cc:224] Handling call
0709 13:17:01.193666 (+ 10737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.326901 (+2133235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.609830 (+1282929us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54003d36d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:04.609996 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4902235392}
W20260709 13:17:04.624348 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.583403 16043 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.637064 16215 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:04.637806 16215 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:04.638406 16215 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:04.628068 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:04.629482 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.632494 16192 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.639549 11549 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:04.639755 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.630185 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:04.630388 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.630911 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.631525 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.631474 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:04.594076 16174 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:04.595855 16220 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.634872 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.605856 15886 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:04.645983 11543 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:04.632696 16001 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.651767 11550 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:04.646335 11546 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:04.657948 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:04.653585 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:04.557979 16235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.020s	sys 0.050s
I20260709 13:17:04.635567 15840 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.616037 16182 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.649365 16311 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:04.639691 16192 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.685477 15807 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 13:17:04.683115 16235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.232s	user 0.021s	sys 0.050s
I20260709 13:17:04.708751 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:04.647902 16324 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:04.687987 16292 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:04.690754 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 73740.5us
I20260709 13:17:04.715365 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:04.716315 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:04.709128 16286 consensus_meta.cc:287] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.744s	user 0.006s	sys 0.006s
I20260709 13:17:04.724574 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.645588 16043 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.728135 11472 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:04.681615 16220 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.699275 16326 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:04.729777 16194 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.719882 16311 raft_consensus.cc:2764] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:17:04.735423 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:04.739616 16192 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.739992 15807 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:04.737953 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 829) took 1130 ms. Trace:
I20260709 13:17:04.738432 16220 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.739255 11542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41380 (request call id 653) took 1453 ms. Trace:
W20260709 13:17:04.739878 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 57723.5us
I20260709 13:17:04.747664 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.745900 11543 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:04.692718 15840 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.687227 16001 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.743237 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:04.742841 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.754773 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 832) took 1146 ms. Trace:
I20260709 13:17:04.758869 11468 rpcz_store.cc:276] 0709 13:17:03.607524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.045973 (+438449us) service_pool.cc:224] Handling call
0709 13:17:04.046427 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.046872 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:17:04.084357 (+ 37485us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.379623 (+295266us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.380106 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.380262 (+   156us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:04.380291 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.380361 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.737697 (+357336us) raft_consensus.cc:1685] finished
0709 13:17:04.737726 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.737887 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207380,"prepare.queue_time_us":37744,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":265038,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":20061,"thread_start_us":87439,"threads_started":1,"wal-append.queue_time_us":603889}
I20260709 13:17:04.759351 11542 rpcz_store.cc:276] 0709 13:17:03.285852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.308262 (+ 22410us) service_pool.cc:224] Handling call
0709 13:17:03.308639 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.387823 (+ 79184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:03.387854 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:03.387895 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:03.387924 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.738861 (+1350937us) spinlock_profiling.cc:243] Waited 79 ms on lock 0x7b54000b2a54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:04.738919 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.739100 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181674880}
I20260709 13:17:04.770838 11474 rpcz_store.cc:276] 0709 13:17:03.608484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.056241 (+447757us) service_pool.cc:224] Handling call
0709 13:17:04.059092 (+  2851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.059474 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.059506 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.059951 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.564205 (+504254us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:04.564247 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.564532 (+   285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.754521 (+189989us) raft_consensus.cc:1685] finished
0709 13:17:04.754541 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.754710 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"thread_start_us":503666,"threads_started":1,"wal-append.queue_time_us":627042}
I20260709 13:17:04.772020 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.772753 11468 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:04.773036 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:04.773556 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.770900 11476 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:04.776757 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.779904 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.780727 11474 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:04.751340 16185 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:04.781826 16185 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.782094 16236 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.782193 16185 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.782310 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:04.782699 16236 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.784770 16302 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:04.785171 16302 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:04.785527 16302 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.770097 16211 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:17:04.735409 16333 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:04.795759 16314 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:04.796175 16314 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:17:04.796409 16142 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.830s	user 0.002s	sys 0.008s
I20260709 13:17:04.796914 16142 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:04.797813 16310 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:04.798271 11473 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:04.792724 16311 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:04.728742 16199 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:17:04.799319 16311 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.747008 15807 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:04.801669 16311 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:04.802667 15807 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:04.804419 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.804301 15807 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:04.806320 16302 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.807155 16220 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:04.808729 16302 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:04.775722 16003 consensus_meta.cc:287] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.908s	user 0.008s	sys 0.004s
I20260709 13:17:04.775110 16260 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:04.827986 16188 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.756727 16255 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:04.729074 16286 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.832355 16001 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.810717 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:04.787679 16006 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.001s
I20260709 13:17:04.842495 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:04.842810 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.843746 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:04.848999 12115 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 15: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:04.845909 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2608) took 1371 ms. Trace:
I20260709 13:17:04.847813 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:04.794646 16333 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.785545 16043 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:04.841933 16263 consensus_meta.cc:287] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.001s
I20260709 13:17:04.854175 11542 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:04.847180 16015 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:04.842530 16003 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.859141 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.859764 11476 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:04.851560 16006 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.860598 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.861289 11476 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 5: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:04.862181 16006 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.857081 16333 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:04.863277 11477 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:04.838337 16199 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.856554 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.866088 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2589) took 2633 ms. Trace:
I20260709 13:17:04.844805 16255 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:04.865661 11396 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:04.866524 11545 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:04.855989 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:04.796986 16314 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:04.851711 16311 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:04.921348 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:04.920558 11471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 843) took 1277 ms (client timeout 1546 ms). Trace:
I20260709 13:17:04.925487 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.903764 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 831) took 1295 ms. Trace:
I20260709 13:17:04.910446 12083 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:04.900312 16192 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:04.975114 11393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 2085) took 2771 ms (client timeout 1850 ms). Trace:
I20260709 13:17:04.910449 16286 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:04.977150 11393 rpcz_store.cc:269] 0709 13:17:02.204243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.204487 (+   244us) service_pool.cc:224] Handling call
0709 13:17:04.974887 (+2770400us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540014aad4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:04.975014 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921368960}
I20260709 13:17:04.864578 15718 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:04.978746 15718 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.979158 15718 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.979722 16192 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:04.980906 15973 consensus_meta.cc:287] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.936s	user 0.007s	sys 0.003s
I20260709 13:17:04.982232 16192 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.985930 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:04.987237 11473 rpcz_store.cc:276] 0709 13:17:03.608203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.053641 (+445438us) service_pool.cc:224] Handling call
0709 13:17:04.114519 (+ 60878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.798763 (+684244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.903412 (+104649us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54004295d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:04.903669 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571944320}
W20260709 13:17:04.988303 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 77934.9us
W20260709 13:17:04.988947 11471 rpcz_store.cc:269] 0709 13:17:03.643370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.682612 (+ 39242us) service_pool.cc:224] Handling call
0709 13:17:04.920311 (+1237699us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b540039b2d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:04.920481 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19910016}
I20260709 13:17:04.989761 12083 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:04.946488 11543 raft_consensus.cc:2393] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:04.991305 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.992856 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:04.899344 16184 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.999s	user 0.010s	sys 0.001s
W20260709 13:17:04.888056 16332 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
W20260709 13:17:04.894091 15751 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.739s	user 0.006s	sys 0.001s
I20260709 13:17:04.996548 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:04.997838 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:05.005336 11549 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:17:05.006230 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.006367 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:04.894999 16140 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.007449 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:04.999047 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:04.905573 16194 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.889400 16182 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.915753 16174 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.919246 16330 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:04.900127 16001 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.009537 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.010263 16314 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:04.990284 16311 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.013819 15886 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:04.865962 16274 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.992624 15807 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:04.880713 16006 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:05.024379 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 103102us
I20260709 13:17:05.041519 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.043066 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.024675 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.046106 11546 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:17:04.866793 16263 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.026945 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:04.894845 16188 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.015344 16192 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:05.027429 16260 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:05.021147 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.027031 16015 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.014230 16140 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.026790 16200 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:05.053961 11549 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.030814 16182 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:05.035116 11473 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:05.027599 16043 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.119458 16043 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.120447 16140 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.007679 16320 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:17:05.027253 16174 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:04.884955 16199 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.035912 16311 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:05.061908 16274 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:05.062114 16006 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.065516 16325 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.168222 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:05.007391 16184 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.076565 11542 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:05.011821 15751 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.088757 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:05.099536 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:05.172509 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 73067.9us
I20260709 13:17:05.020936 16194 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:05.108492 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.128444 11516 connection.cc:256] RPC call timeout handler was delayed by -2.5605323760000829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260709 13:17:05.075376 16192 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.185897 16263 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.186475 16192 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:05.188267 16280 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 1.819s	user 0.001s	sys 0.005s
I20260709 13:17:05.072723 16324 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.063194 16337 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:05.188827 16324 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.075534 16188 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.122561 16182 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.195043 16182 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:05.196367 15751 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.197686 16006 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.198206 16194 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.198627 16274 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:05.062793 16231 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 13:17:05.100325 16015 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.199676 16274 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.034341 16001 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:05.178587 11516 connection.cc:256] RPC call timeout handler was delayed by -2.3922323760000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260709 13:17:05.078527 16333 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.181967 16174 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.182585 16345 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.225188 16345 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.226404 16333 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.227582 16345 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.183001 15973 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.228468 15973 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.037606 16255 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.185832 16184 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.187848 16185 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.065879 16327 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:05.231060 16255 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:05.206423 16043 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.230799 15973 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.232951 16185 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.234014 11546 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:17:05.201746 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.202020 11545 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.132546 16260 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:05.237259 16188 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.239353 11546 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:05.240466 11399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 701) took 1982 ms. Trace:
I20260709 13:17:05.240980 16001 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.240854 11399 rpcz_store.cc:276] 0709 13:17:03.257656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.257839 (+   183us) service_pool.cc:224] Handling call
0709 13:17:03.277993 (+ 20154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:03.312118 (+ 34125us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:17:03.312433 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.358683 (+ 46250us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:03.359301 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:17:03.359521 (+   220us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:03.359555 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.914056 (+1554501us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b54003d40d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:04.914114 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.188005 (+273891us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b2400137078. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686b09d 00007f8da949d10e 00007f8da94b488c 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221
0709 13:17:05.240264 (+ 52259us) raft_consensus.cc:1685] finished
0709 13:17:05.240284 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.240411 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54,"prepare.queue_time_us":824,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":6914,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":91761,"spinlock_wait_cycles":153372416,"thread_start_us":45940,"threads_started":1,"wal-append.queue_time_us":44975}
I20260709 13:17:05.240494 11546 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:05.245273 16232 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:05.201095 16015 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:05.204921 11516 connection.cc:256] RPC call timeout handler was delayed by -2.1075740809999388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 13:17:05.246136 11516 connection.cc:256] RPC call timeout handler was delayed by -2.0475323759999355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
W20260709 13:17:05.246479 11516 connection.cc:256] RPC call timeout handler was delayed by -1.9944323760000771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 13:17:05.246757 11516 connection.cc:256] RPC call timeout handler was delayed by -1.9575323760000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260709 13:17:05.247038 11516 connection.cc:256] RPC call timeout handler was delayed by -1.8387310589998833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:17:05.247327 11516 connection.cc:256] RPC call timeout handler was delayed by -1.7546916559999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 13:17:05.247602 11516 connection.cc:256] RPC call timeout handler was delayed by -1.7429916559999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 13:17:05.247879 11516 connection.cc:256] RPC call timeout handler was delayed by -1.6134715189998587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:17:05.248159 11516 connection.cc:256] RPC call timeout handler was delayed by -1.5462323760000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2631s for a response
W20260709 13:17:05.248435 11516 connection.cc:256] RPC call timeout handler was delayed by -1.4764938299999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 13:17:05.248695 11516 connection.cc:256] RPC call timeout handler was delayed by -1.4625481030000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
W20260709 13:17:05.248953 11516 connection.cc:256] RPC call timeout handler was delayed by -1.452193829999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 13:17:05.203343 16175 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.212112 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.217607 16246 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:05.220125 16301 consensus_meta.cc:287] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
I20260709 13:17:05.125938 16215 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.179875 16140 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:05.223920 16233 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:05.270794 11516 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:05.271202 16233 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:05.245354 16199 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:04.866466 11470 rpcz_store.cc:276] 0709 13:17:02.232361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.518371 (+286010us) service_pool.cc:224] Handling call
0709 13:17:02.518658 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:03.123964 (+605306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:03.124014 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:03.152916 (+ 28902us) log.cc:844] Serialized 52 byte log entry
0709 13:17:04.785724 (+1632808us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400551ca0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da98906b8 00007f8da991992b 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c
0709 13:17:04.785785 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:04.786066 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.854126 (+ 68060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.854240 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.854300 (+    60us) raft_consensus.cc:1685] finished
0709 13:17:04.865429 (+ 11129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.866040 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600725,"prepare.queue_time_us":1163,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":607870,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":32203,"spinlock_wait_cycles":243392256,"thread_start_us":247248,"threads_started":2,"wal-append.queue_time_us":287517,"wal-append.run_cpu_time_us":1939,"wal-append.run_wall_time_us":1145267}
I20260709 13:17:05.272119 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.245177 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.031787 16330 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:05.254781 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 248866us
I20260709 13:17:05.249230 16143 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:17:05.223978 15718 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.261608 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2583) took 2561 ms. Trace:
W20260709 13:17:05.265554 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.278340 16015 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:05.278575 11550 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:05.280118 16182 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.179224 16325 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.269991 16140 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.271420 16301 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.268239 16175 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.273236 16233 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.276155 12112 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:05.260447 16246 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.282562 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.291381 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:05.288540 16326 consensus_meta.cc:287] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
I20260709 13:17:05.284848 16043 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:05.286412 16325 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.282248 16232 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:05.287513 16140 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:05.302903 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:05.306274 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.264633 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 271512us
I20260709 13:17:05.290805 16324 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.292658 16185 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:05.316154 16006 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.316682 16006 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.318712 16006 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.304028 16232 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:05.301683 16174 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.307543 15886 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:05.331229 15080 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.303902 16327 consensus_queue.cc:1059] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:17:05.333117 16327 raft_consensus.cc:3055] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:05.333473 16327 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:05.303926 16188 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.335580 16188 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.336112 16174 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.299753 16001 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.304631 16192 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.301796 16326 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:05.334359 16327 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.338924 16174 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.341347 16246 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.343897 16330 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.356114 16301 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.330036 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:05.307029 16233 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:05.303272 16347 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:05.308094 11516 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:05.474114 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 166088us
W20260709 13:17:05.456243 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 126333us
I20260709 13:17:05.456686 11472 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:05.475317 16043 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:05.339092 16188 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.344319 16182 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.339931 16215 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.491969 16327 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.453225 16301 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.356966 16140 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.477994 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.478322 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:05.494035 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.479121 11473 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:05.469345 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 171895us
I20260709 13:17:05.343740 15718 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.482146 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.498252 11472 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:05.499173 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.499735 11472 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:05.470935 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.501000 11468 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:05.480621 16180 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:05.486230 16192 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.342442 16232 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.454017 16001 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.475193 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:05.511489 11516 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:05.512447 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.473961 16350 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:05.520538 11516 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.532572 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:05.535399 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:17:05.536597 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 263714us
I20260709 13:17:05.538206 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.538530 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:05.538952 12118 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:05.539783 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.540207 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.541903 16185 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:05.543308 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.544119 12123 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:05.534790 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.545445 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.546815 11472 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:05.548319 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.549976 12114 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:05.548493 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:05.523836 16043 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:05.557387 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.513446 16020 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:05.523418 15638 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.514911 16347 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:05.568873 15638 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.514406 16301 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.571677 15638 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.559186 11441 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:05.573915 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.556867 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.531014 16350 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.534915 15718 raft_consensus.cc:2764] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:17:05.563200 11472 raft_consensus.cc:2410] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:17:05.566489 11468 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:05.580689 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.581399 11472 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:17:05.581727 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.582449 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.494762 16311 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.582481 11468 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:05.558149 16119 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:17:05.558044 16270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.845s	user 0.010s	sys 0.024s
I20260709 13:17:05.577741 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.522327 16180 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.566578 16232 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:05.567454 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 53076.2us
I20260709 13:17:05.607676 11476 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:05.580374 16181 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:05.576805 16347 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.580155 16350 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.579483 16175 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.564284 16357 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:05.575865 16192 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:05.622491 16192 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:05.586318 16119 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:05.596398 16311 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:05.597273 11546 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:05.597343 16185 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:05.583278 16020 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 13:17:05.602017 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.602418 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.571794 16301 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.604266 12083 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:17:05.652180 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47989.7us
I20260709 13:17:05.653777 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:05.654071 16355 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:05.604794 16142 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.604858 16334 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:17:05.606253 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.566417 15973 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:05.585850 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.610251 11473 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:05.722759 11468 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:17:05.587248 16270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.919s	user 0.011s	sys 0.024s
I20260709 13:17:05.590497 16140 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.575830 16004 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.625730 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:04.852268 11548 rpcz_store.cc:276] 0709 13:17:03.474926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.476665 (+  1739us) service_pool.cc:224] Handling call
0709 13:17:03.480297 (+  3632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.480493 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:04.808636 (+1328143us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b180044e420. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da94c14bc 00007f8da680f48b 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da6829455 00007f8dad3113c7 00007f8da99239a7 00007f8da99330bf
0709 13:17:04.808797 (+   161us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.845504 (+ 36707us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:04.845559 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:04.845603 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.845743 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.845853 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10194,"prepare.queue_time_us":99765,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":22228,"raft.queue_time_us":2025,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2700,"spinlock_wait_cycles":22054144,"thread_start_us":95142,"threads_started":2}
I20260709 13:17:05.277316 11544 rpcz_store.cc:276] 0709 13:17:02.699748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.699934 (+   186us) service_pool.cc:224] Handling call
0709 13:17:02.708640 (+  8706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:02.729120 (+ 20480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.729178 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:02.736862 (+  7684us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.914703 (+2177841us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b180044e540. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:17:05.021054 (+106351us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:05.201641 (+180587us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.203105 (+  1464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.204684 (+  1579us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.252126 (+ 47442us) raft_consensus.cc:1685] finished
0709 13:17:05.261353 (+  9227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.261519 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39278,"prepare.queue_time_us":248978,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":22210,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15498,"spinlock_wait_cycles":37278464,"thread_start_us":270147,"threads_started":2,"wal-append.queue_time_us":2414715}
I20260709 13:17:05.627537 11551 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.630429 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.633044 11469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 839) took 1991 ms. Trace:
I20260709 13:17:05.635597 11365 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:05.650744 11471 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:05.656754 12114 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:05.736508 11469 rpcz_store.cc:276] 0709 13:17:03.642063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.133312 (+491249us) service_pool.cc:224] Handling call
0709 13:17:04.133688 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.595089 (+461401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.595137 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:04.595199 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:04.595245 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.632699 (+1037454us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540028a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:05.632759 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.632967 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060495232}
I20260709 13:17:05.723651 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.639032 16211 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:17:05.737507 12114 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:05.629225 16180 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.644311 16192 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:05.630766 16001 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.623296 16357 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.738101 11468 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:17:05.739718 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 104205us
I20260709 13:17:05.740980 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.741829 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:05.654500 16355 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.743136 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.625113 16308 consensus_queue.cc:1059] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:17:05.626948 16119 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.739140 12114 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.776652 12114 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:05.620713 16347 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.776014 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43430 (request call id 2585) took 3035 ms. Trace:
I20260709 13:17:05.779101 11544 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:05.782704 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.778869 11548 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:05.733328 16270 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:05.780366 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:05.781059 11551 rpcz_store.cc:276] 0709 13:17:02.720791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.867376 (+146585us) service_pool.cc:224] Handling call
0709 13:17:05.627019 (+2759643us) spinlock_profiling.cc:243] Waited 956 us on lock 0x7b4c004ea240. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da7be5a20 00007f8da7bf6a29 00007f8da7bf713f 00007f8da7bbb2ab 00007f8da7be8f3c 00007f8da7be93fe 00007f8da7e7688b 00007f8da9875495 00007f8da98c630f 00007f8da9910a55 00007f8dae8e0cd0
0709 13:17:05.736114 (+109095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2199296}
I20260709 13:17:05.808401 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:05.809815 16358 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:05.811530 15840 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.811658 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.812466 11470 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 11: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:05.813309 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.814052 11470 raft_consensus.cc:2393] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 8: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:05.738900 16142 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:05.787710 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:05.820590 11548 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.809751 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.778074 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:05.817083 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:05.777628 11473 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:05.818145 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.822559 11550 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:05.825893 11471 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:05.826893 11470 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:05.828330 16020 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:05.829357 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:05.829747 11551 raft_consensus.cc:2393] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.829996 11543 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:05.830235 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.744881 16004 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:05.834077 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:05.834285 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:05.831676 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.835418 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:05.776757 16355 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:05.842581 11544 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:17:05.834861 12119 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:05.845022 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.848762 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.849186 11548 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:05.849452 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:05.849974 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.655215 16350 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.851629 16020 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:05.893714 16308 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:05.893165 16020 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.903033 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.903903 11546 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:05.903574 11543 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:05.905915 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.906770 11547 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 12: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.909344 11547 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:05.910370 11543 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:17:05.925809 16350 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:05.837545 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.883752 15840 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:05.846906 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:05.929787 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92325.8us
I20260709 13:17:05.850705 11548 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:05.930621 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.836045 16142 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.858129 11468 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:05.783464 16357 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.855240 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.867439 16270 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:05.870005 16347 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:05.887811 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 57697.4us
I20260709 13:17:05.936158 11365 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:05.890697 12114 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:05.850013 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:05.911244 16186 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:05.891376 12113 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 5.
W20260709 13:17:05.919063 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.853739 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.937171 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:05.851462 16185 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:05.932698 11472 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:05.930159 11551 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:05.933868 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:05.910709 11547 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:17:05.935161 11550 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:05.941078 11546 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:05.951382 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.929888 15840 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.955689 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.962415 11472 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:17:05.967160 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.939746 16001 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:05.939582 16174 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:05.944795 16275 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:05.968815 15840 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:05.975018 16254 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:05.969594 16301 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:05.945916 16185 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.953851 16182 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:06.015935 16185 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:05.973366 16188 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:06.021880 16188 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.022343 16188 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:05.973029 16001 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.031124 16001 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:06.040432 16233 consensus_meta.cc:287] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.567s	user 0.002s	sys 0.007s
I20260709 13:17:05.972577 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:05.972527 16292 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:05.946044 16232 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:17:05.930701 16355 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:05.981032 16186 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:05.981381 16174 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.015198 16301 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.069389 16142 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.012720 16275 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:06.070614 16340 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.171s	user 0.009s	sys 0.021s
I20260709 13:17:06.013422 16254 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.093659 16254 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.073699 16274 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.094329 16274 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.096386 16274 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.097061 16142 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:06.097478 16142 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.097816 16142 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.098287 15840 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.089378 16301 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.077229 16188 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.078917 16185 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.061355 16292 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.088006 16174 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.091253 16275 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.089603 16186 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:06.104378 16236 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.104761 16185 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.111585 16186 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.104969 16292 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:06.120662 16357 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.130512 16292 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.131012 16301 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.131644 16001 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.111266 16188 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.114784 16142 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.141899 12115 raft_consensus.cc:2368] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
I20260709 13:17:06.155949 16357 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.177932 16355 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.179231 16292 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.178871 16301 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:06.215375 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.223986 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.227123 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:06.230086 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.229961 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:06.236091 11471 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:06.241932 16001 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.243508 16020 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:06.245489 15840 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.244870 16175 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:06.246846 16175 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:06.247535 16070 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.248575 16070 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.249028 16175 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.251927 16175 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:06.244860 16020 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.241961 16142 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.256747 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 710) took 2145 ms. Trace:
I20260709 13:17:06.257331 11395 rpcz_store.cc:276] 0709 13:17:04.110655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.111590 (+   935us) service_pool.cc:224] Handling call
0709 13:17:04.111862 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.508137 (+396275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.256025 (+1747888us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540030aa54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:06.256444 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878385024}
I20260709 13:17:06.267616 16070 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.267884 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.268373 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2624) took 1555 ms. Trace:
W20260709 13:17:06.273667 11545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43442 (request call id 2169) took 1299 ms (client timeout 1565 ms). Trace:
W20260709 13:17:06.274262 11545 rpcz_store.cc:269] 0709 13:17:04.974130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.994906 (+ 20776us) service_pool.cc:224] Handling call
0709 13:17:06.273108 (+1278202us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400404f54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:06.273415 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440812800}
I20260709 13:17:06.275166 16192 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:06.275820 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.265908 16020 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.276598 16207 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:06.279017 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:06.328655 16200 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 13:17:06.364382 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 391746us
I20260709 13:17:06.366343 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:06.367125 11549 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:06.367676 16142 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:06.368176 16142 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.368615 16142 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:06.369230 11549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2628) took 1355 ms. Trace:
I20260709 13:17:06.369755 11549 rpcz_store.cc:276] 0709 13:17:05.014042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.262768 (+248726us) service_pool.cc:224] Handling call
0709 13:17:06.279344 (+1016576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.367838 (+ 88494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.368858 (+  1020us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400404f50. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da99188ad 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:06.369100 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2327939456}
I20260709 13:17:06.387306 16182 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:06.389461 16182 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.273123 11477 rpcz_store.cc:276] 0709 13:17:04.713086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.978476 (+265390us) service_pool.cc:224] Handling call
0709 13:17:04.978749 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.178153 (+199404us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:05.201427 (+ 23274us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.204927 (+  3500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:05.204981 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:05.205022 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.267921 (+1062899us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400213254. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:06.267985 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.268181 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3242,"prepare.queue_time_us":4780,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":3596,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":458053888}
I20260709 13:17:06.389490 16003 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:06.411505 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.415997 16192 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.423241 16372 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.426586 16372 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:06.425782 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.431106 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50624.5us
I20260709 13:17:06.432026 16192 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:06.435880 16192 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.437412 11469 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:06.437986 16372 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.438688 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.438241 16192 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.462327 16175 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.464550 16175 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.465607 12114 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:06.466636 16180 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:06.467203 16180 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:06.468938 11545 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:06.468698 16180 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:06.469470 16371 log.cc:927] Time spent T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 13:17:06.468997 16307 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:17:06.471205 16307 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:06.471662 16307 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:06.472986 16307 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.474237 16307 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:06.485071 15630 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:06.485597 16325 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:06.486126 15630 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.487304 16320 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:06.487421 16235 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:06.493597 16235 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:17:06.493989 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.494159 15630 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.494026 16260 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.496806 16260 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.497699 11547 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:06.498610 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.498867 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:17:06.499392 16369 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.110s	user 0.002s	sys 0.002s
I20260709 13:17:06.499666 11547 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:06.500185 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.496817 11546 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:06.488073 16321 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:06.443387 12112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.044s	sys 0.088s
I20260709 13:17:06.489302 16305 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:17:06.470727 16180 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:06.499002 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.511732 11551 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:06.508701 12112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.044s	sys 0.088s
I20260709 13:17:06.520561 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.525650 11441 leader_election.cc:400] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
I20260709 13:17:06.526072 11441 leader_election.cc:403] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:17:06.491899 16320 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:06.494033 16235 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:06.527361 16186 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:06.512403 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.583902 11548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2640) took 1331 ms. Trace:
W20260709 13:17:06.599552 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 107663us
W20260709 13:17:06.610572 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 85358.8us
I20260709 13:17:06.611660 11545 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13.
I20260709 13:17:06.613107 16190 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:06.613659 16190 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:06.636962 11474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2623) took 1924 ms. Trace:
I20260709 13:17:06.637096 16235 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.583953 16321 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.637641 11474 rpcz_store.cc:276] 0709 13:17:04.712770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.930805 (+218035us) service_pool.cc:224] Handling call
0709 13:17:04.943462 (+ 12657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.573030 (+629568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.574346 (+  1316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:05.574392 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:05.574426 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.636347 (+1061921us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54002a4c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:06.636446 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.636727 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1447414016}
I20260709 13:17:06.641621 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.642452 11549 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
W20260709 13:17:06.643321 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.643779 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.645749 16238 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.646436 16238 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.649215 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.649428 16238 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.650148 11549 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:06.651679 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.651551 16142 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.653019 16270 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:06.653761 16142 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.654256 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:06.654963 12114 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:06.655385 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.659871 16320 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.661218 16320 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:06.663136 16294 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.663538 16221 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:06.664649 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:06.664965 16321 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.665482 11515 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:06.666531 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:06.668697 11550 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:06.667337 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.669127 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:17:06.670147 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 167326us
I20260709 13:17:06.654762 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.600843 11548 rpcz_store.cc:276] 0709 13:17:05.176082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.930796 (+754714us) service_pool.cc:224] Handling call
0709 13:17:05.931093 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.931290 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:06.141320 (+210030us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.432455 (+291135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:06.447366 (+ 14911us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:06.491281 (+ 43915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.502175 (+ 10894us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.507711 (+  5536us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":237278,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1733,"raft.queue_time_us":821,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":286,"thread_start_us":484610,"threads_started":2,"wal-append.queue_time_us":441003}
W20260709 13:17:06.695425 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.681027 11542 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:06.696221 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2641) took 1452 ms. Trace:
I20260709 13:17:06.696707 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.617475 16175 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.697969 11542 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:06.699457 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:06.701351 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:06.707055 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:06.708521 11544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2644) took 1255 ms. Trace:
I20260709 13:17:06.709937 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.696774 11543 rpcz_store.cc:276] 0709 13:17:05.243644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.943834 (+700190us) service_pool.cc:224] Handling call
0709 13:17:05.969965 (+ 26131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:05.973462 (+  3497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.973508 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.974152 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:17:06.435412 (+461260us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:06.435508 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.435633 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.468650 (+ 33017us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:06.695678 (+227028us) raft_consensus.cc:1685] finished
0709 13:17:06.695707 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.696027 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46811904,"thread_start_us":418839,"threads_started":1,"wal-append.queue_time_us":415430}
I20260709 13:17:06.750368 12113 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.752766 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.711628 11544 rpcz_store.cc:276] 0709 13:17:05.251440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.942840 (+691400us) service_pool.cc:224] Handling call
0709 13:17:05.943213 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:05.946370 (+  3157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.946407 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.947039 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 13:17:06.254602 (+307563us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b180044eae0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:17:06.430167 (+175565us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:06.432008 (+  1841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.434752 (+  2744us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.490230 (+ 55478us) raft_consensus.cc:1685] finished
0709 13:17:06.490270 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.506770 (+ 16500us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2675,"raft.queue_time_us":2784,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5525,"spinlock_wait_cycles":6850432,"thread_start_us":121421,"threads_started":1,"wal-append.queue_time_us":532592}
I20260709 13:17:06.691352 16221 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.763723 15080 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.764238 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.764482 15080 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.765127 11543 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:06.764642 16190 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.766027 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.766469 16190 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.749828 16175 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:06.709287 16294 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.766904 11543 raft_consensus.cc:2393] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:06.790153 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:06.766722 15080 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.773439 16321 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.806824 16311 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:06.809065 16311 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.809509 16311 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:06.764256 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:06.810439 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:06.789013 11544 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:06.787976 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:06.812641 11399 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 10: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:06.812961 16260 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.806885 16175 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:06.815927 16175 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:06.777886 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.817689 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:06.819096 16174 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.014s
I20260709 13:17:06.775671 15630 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:17:06.819625 16174 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:06.820269 16260 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:06.821758 16294 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:06.773875 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:06.827503 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 187468us
I20260709 13:17:06.829660 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:06.830359 11543 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:06.830565 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.832499 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:06.833137 11542 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:06.842037 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 87492.4us
I20260709 13:17:06.844801 15080 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:06.850979 15080 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.851387 15080 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.830808 11544 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:06.831176 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:06.851432 11475 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43640 (request call id 1778) took 3289 ms (client timeout 1755 ms). Trace:
I20260709 13:17:06.857141 11551 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:17:06.882124 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:06.882895 11542 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:06.787882 16230 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:06.883812 16230 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:06.887283 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 851) took 1290 ms. Trace:
I20260709 13:17:06.887903 11468 rpcz_store.cc:276] 0709 13:17:05.596370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.950566 (+354196us) service_pool.cc:224] Handling call
0709 13:17:05.950849 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.951051 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:06.448812 (+497761us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.747022 (+298210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:06.756509 (+  9487us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:06.761470 (+  4961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.769929 (+  8459us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.887078 (+117149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":489084,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":6427,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":38039,"thread_start_us":574405,"threads_started":2}
W20260709 13:17:06.800004 16376 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.409s	user 0.002s	sys 0.014s
I20260709 13:17:06.891620 11547 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
W20260709 13:17:06.776533 16375 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.389s	user 0.003s	sys 0.014s
W20260709 13:17:06.854795 11475 rpcz_store.cc:269] 0709 13:17:03.562124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.562274 (+   150us) service_pool.cc:224] Handling call
0709 13:17:06.769795 (+3207521us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b540039a3d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:06.851204 (+ 81409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6766984960}
I20260709 13:17:06.821774 16174 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:06.778975 16235 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:06.939844 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 335890us
W20260709 13:17:06.973090 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 322593us
I20260709 13:17:07.001211 11545 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:07.000829 12118 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:07.008730 12114 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:07.016585 12113 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
W20260709 13:17:07.016166 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 116696us
W20260709 13:17:07.017758 16307 consensus_meta.cc:287] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.015s
I20260709 13:17:07.036573 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:07.050170 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:17:07.052273 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.053122 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.054340 11474 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:07.056089 11544 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:07.056497 11544 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:07.051551 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.061136 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.061300 16277 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:07.062006 16277 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:07.063483 16277 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.065507 16277 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:07.067147 15807 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.067776 15807 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.071272 11549 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:17:07.073683 11398 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:07.078809 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:07.024181 16175 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.042107 16230 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:07.086536 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:07.098394 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.098430 11472 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:07.089141 11546 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.028751 16311 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.069876 15807 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:07.102725 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 43737.1us
W20260709 13:17:07.087667 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.099284 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:17:07.104156 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.109579 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:07.110762 11549 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:07.111272 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 70125.9us
I20260709 13:17:07.107085 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:07.119320 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 297285us
I20260709 13:17:07.117065 11551 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.127784 16221 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.127908 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.129590 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.121474 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.140127 11544 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.147708 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.155143 11470 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:17:07.157986 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:07.158980 15080 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.160578 11545 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:07.162959 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:07.163628 16270 consensus_meta.cc:287] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.001s
I20260709 13:17:07.163529 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.167461 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.164403 11514 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.169644 16181 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.170305 16181 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.171371 11545 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:07.171664 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.172359 11542 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:07.173754 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:07.174001 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 48852us
I20260709 13:17:07.169080 15080 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:07.174429 11544 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:07.180045 11549 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:07.183465 11543 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:17:07.187755 11551 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:07.189895 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.164182 16270 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12.
I20260709 13:17:07.194444 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.200608 11393 raft_consensus.cc:2368] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
I20260709 13:17:07.208627 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:07.211129 11469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.004s	sys 0.027s
W20260709 13:17:07.211659 11469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.004s	sys 0.027s
I20260709 13:17:07.185082 16181 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.222575 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:07.226400 11399 raft_consensus.cc:2410] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:17:07.226503 11547 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:07.193643 16311 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:07.206246 16260 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:07.234926 11396 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.237413 11474 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:07.240092 11398 raft_consensus.cc:2410] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:07.218307 16142 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.186900 15807 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:17:07.244371 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:07.251578 16382 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:17:07.261006 11471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 855) took 1374 ms. Trace:
I20260709 13:17:07.261480 11471 rpcz_store.cc:276] 0709 13:17:05.886808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.237296 (+350488us) service_pool.cc:224] Handling call
0709 13:17:06.259096 (+ 21800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.260287 (+  1191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:06.260330 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:06.475307 (+214977us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b2c0009dd58. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:17:06.475919 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 13:17:06.708137 (+232218us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:06.708197 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.708287 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:07.260653 (+552366us) raft_consensus.cc:1685] finished
0709 13:17:07.260677 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.260938 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":246,"spinlock_wait_cycles":50568960,"thread_start_us":231770,"threads_started":1,"wal-append.queue_time_us":706250}
I20260709 13:17:07.263581 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:07.264840 11471 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.271273 16311 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:07.271584 16260 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.272230 16311 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.275394 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:07.275636 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.221899 16264 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.276465 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.276886 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.276772 16264 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.277567 11395 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:07.276280 11398 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:07.279098 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.279526 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.279688 11395 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
W20260709 13:17:07.280278 16381 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:17:07.280963 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.281672 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.281840 11398 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.281791 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.282786 11396 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:07.284207 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:07.285380 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.281086 16175 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.281035 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.245568 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.291044 11473 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:07.294184 11399 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:07.310132 11546 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
W20260709 13:17:07.299584 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57127.1us
I20260709 13:17:07.311640 11544 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:17:07.296111 11469 raft_consensus.cc:2368] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:07.314083 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.315656 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.316313 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:07.317397 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.318055 11469 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:17:07.318890 11471 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:07.319089 11477 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:17:07.278892 16264 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:07.320775 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.288837 16311 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.324267 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.285863 16142 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.324950 16325 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:07.338438 11399 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:07.324985 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.327907 16390 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:07.355866 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.357875 11402 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:17:07.358049 16182 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.358899 16182 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.362460 16174 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.363240 16174 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.365744 16174 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:07.366545 16182 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.375602 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:07.377729 11544 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:07.383985 16325 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:07.412619 11396 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:07.416994 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:07.419306 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.421177 11474 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:07.423934 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.424791 11399 raft_consensus.cc:2368] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:07.429697 11469 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:07.424647 11475 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:07.445998 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:07.447289 11474 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:17:07.454068 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 311904us
I20260709 13:17:07.460928 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:07.462450 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 364551us
W20260709 13:17:07.463848 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 236296us
I20260709 13:17:07.464043 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.464905 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:07.466970 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 154351us
I20260709 13:17:07.461516 11544 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:07.422644 16390 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:07.477526 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:07.478624 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.478900 11399 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:07.480332 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.481036 11551 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:07.481591 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.481489 15080 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.482203 11549 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:07.482318 15080 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.486109 11475 raft_consensus.cc:1240] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:07.487496 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.483938 11514 leader_election.cc:400] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:07.492589 11514 leader_election.cc:403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:07.498443 16370 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.499199 16370 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.502351 11475 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:07.502514 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.512279 16119 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:07.520606 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51124.3us
I20260709 13:17:07.481138 16182 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:07.516238 11468 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:07.529911 15080 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:07.522825 11474 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:07.533439 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.534293 11399 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.515733 16264 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:07.539664 11471 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:07.541229 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.544723 11396 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:07.546267 16142 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:07.551115 16391 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.551851 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.552520 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.553164 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.553508 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.553901 11396 raft_consensus.cc:2368] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
I20260709 13:17:07.554330 11394 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 18: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.554837 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.555676 11398 raft_consensus.cc:2376] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8. Re-sending same reply.
W20260709 13:17:07.555832 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 60055.6us
I20260709 13:17:07.556545 16395 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.556895 12085 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13.
I20260709 13:17:07.557289 12085 leader_election.cc:403] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:07.541141 16264 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:07.557987 15807 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13.
I20260709 13:17:07.558353 16264 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.552613 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.559347 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.559415 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.552039 16399 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.559736 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.560400 11396 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:17:07.561159 11514 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.562675 11468 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:17:07.565174 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.558861 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:07.568907 11474 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:07.574342 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.576565 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.580416 16264 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:07.536900 16311 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:07.562361 15807 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:17:07.577136 16175 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:07.597070 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.583608 16238 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:07.557143 16395 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.574460 16321 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:17:07.583153 15630 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:07.610658 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 147424us
I20260709 13:17:07.551875 16391 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:07.618191 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 127866us
I20260709 13:17:07.598711 16175 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:07.602022 16238 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.638861 16238 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.575518 16399 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.627251 16391 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:07.627736 16254 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:07.645390 11472 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:07.628695 11476 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:07.629731 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:07.602833 16390 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.636909 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 852) took 1983 ms. Trace:
I20260709 13:17:07.600692 16182 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.641775 16405 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:07.646654 11473 rpcz_store.cc:276] 0709 13:17:05.653273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.137180 (+483907us) service_pool.cc:224] Handling call
0709 13:17:06.137584 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:07.310971 (+1173387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.636661 (+325690us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400388bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:07.636854 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639065344}
I20260709 13:17:07.642689 11399 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:07.617161 16311 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:07.649359 16311 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:07.649549 11365 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.650368 16311 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.652225 16395 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.654699 16182 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.658663 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.659502 11400 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13.
I20260709 13:17:07.659439 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.660562 11395 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:07.660775 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.661305 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.640676 16175 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:07.661913 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.662382 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.663029 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.663342 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.663416 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.664640 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.664979 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.665822 11398 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:17:07.666005 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.666591 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.664628 11401 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:07.668467 11398 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:17:07.661587 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.663733 11400 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:17:07.676445 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54057.3us
I20260709 13:17:07.677619 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.643545 16176 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:07.679385 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:07.679786 16176 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:07.680430 15630 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:07.680725 16176 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.681957 16264 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:07.682171 16176 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.682723 16264 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:07.683667 11550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2646) took 1180 ms. Trace:
I20260709 13:17:07.683728 16264 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.684157 11550 rpcz_store.cc:276] 0709 13:17:06.503612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.759996 (+256384us) service_pool.cc:224] Handling call
0709 13:17:06.852802 (+ 92806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:07.021376 (+168574us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:07.414568 (+393192us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.682941 (+268373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:07.683384 (+   443us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:07.683428 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.683494 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.683610 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":362256,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":26988,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11248,"spinlock_wait_cycles":9243392,"thread_start_us":528193,"threads_started":3,"wal-append.queue_time_us":190217}
I20260709 13:17:07.677636 16405 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:07.687106 16235 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:07.687351 16405 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.690229 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:17:07.703092 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 134372us
I20260709 13:17:07.707952 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:07.712118 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.713043 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.714193 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.712335 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.715649 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.716033 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.716248 11401 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:07.716989 11399 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.717356 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.718102 11402 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.718024 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.714118 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.713143 11395 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:07.719924 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.720130 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.720499 11395 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:07.721664 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.726310 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.719117 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.724237 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:07.728091 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51039.9us
I20260709 13:17:07.728466 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.733167 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.733999 11400 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 17: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.734771 11395 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:07.730891 16182 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:07.737792 16124 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:07.737932 16182 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:07.738812 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62314.9us
I20260709 13:17:07.740589 11367 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.740721 16182 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.741420 16235 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:07.742966 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:07.743842 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 53767.1us
I20260709 13:17:07.744597 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.746304 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:07.755924 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 79630.8us
I20260709 13:17:07.760864 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.761861 11401 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:07.765738 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.747314 16235 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:07.759706 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:07.773840 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.779860 11367 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.748095 16124 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:07.767433 15807 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:07.790383 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:07.801951 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57249.7us
I20260709 13:17:07.799876 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:07.809443 16140 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:07.809641 16124 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.812044 15099 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:07.812527 15099 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.812901 15099 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.813319 16336 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:07.813805 16336 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.814198 16336 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:07.815829 16182 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:07.816246 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.817154 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:07.817351 16182 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:07.817867 16182 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:07.818274 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.813179 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:07.827124 11393 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:07.839463 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:07.840334 12112 raft_consensus.cc:2393] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:07.840862 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:07.842696 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:07.819156 16174 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:07.844843 11441 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:07.861661 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.832537 16142 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:07.867388 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:07.868408 11515 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:07.875167 16119 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:07.882043 16411 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:07.875655 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:07.863279 16174 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:07.874580 16277 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.009s
W20260709 13:17:07.887871 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 74806.7us
I20260709 13:17:07.849272 15630 consensus_queue.cc:1059] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:17:07.891016 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:17:07.891669 16414 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:17:07.893283 16142 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:07.893996 16142 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:07.894341 16142 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:07.894724 16142 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:07.899345 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:07.901564 16413 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:17:07.902624 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.818887 16182 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.914004 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:07.905869 16370 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:07.883356 16311 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:07.888399 16174 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:07.920817 16404 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.921197 16425 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:07.921597 16425 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:07.921473 16404 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.922536 16425 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.923944 16425 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:07.906281 16399 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.888517 16175 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:07.928165 16277 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.928689 16277 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:07.929450 16175 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:07.930661 16277 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:07.931648 16321 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:07.932665 16311 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:07.885320 15807 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:07.881060 16235 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:07.995141 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46282.4us
W20260709 13:17:07.995281 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 198142us
W20260709 13:17:07.999122 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.958158 16418 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:08.003332 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 185118us
W20260709 13:17:07.997051 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 106154us
I20260709 13:17:07.987915 16221 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:17:07.991175 16399 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.008955 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.009217 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.009943 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:08.005162 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 94605.2us
I20260709 13:17:08.004514 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:08.006899 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.016170 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.017356 11402 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:08.018702 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.027308 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.028729 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.021836 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:08.010941 16421 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.033034 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.036095 11516 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.029502 12115 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:17:08.031456 15099 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:08.038653 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 139442us
I20260709 13:17:08.002539 16238 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.035953 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.041304 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.003252 16254 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:17:08.042554 12123 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:08.038486 16421 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.048113 16421 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.019976 15630 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.043435 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.017078 16427 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.051589 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:08.017251 16418 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.057909 16336 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.059178 15630 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.078786 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.063429 16432 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:08.079496 16437 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:08.079730 16432 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:08.079732 11394 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:08.068674 16238 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.083623 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.069329 11395 raft_consensus.cc:2368] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:17:08.073220 15099 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.076051 11398 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
W20260709 13:17:08.076915 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 57340.5us
I20260709 13:17:08.077214 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.064038 16399 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:08.085283 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.040932 15807 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.087690 15705 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:08.088757 16432 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.089965 16418 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.090503 12113 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 5.
I20260709 13:17:08.091142 16336 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.091853 16142 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.080152 16437 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:08.110762 16437 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:08.093179 16403 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.511s	user 0.002s	sys 0.002s
I20260709 13:17:08.097323 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:08.098208 11469 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:08.098656 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:08.113127 11468 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.099747 11400 raft_consensus.cc:2410] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:08.055636 16404 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.100839 11399 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:08.105427 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.105614 11472 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:08.039264 16311 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.106441 12085 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13.
I20260709 13:17:08.106948 16320 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:08.120141 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:08.120929 11471 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:08.104104 11442 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:08.114430 11401 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:08.116525 11394 raft_consensus.cc:2410] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:17:08.116969 16399 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:08.117285 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.117511 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.125084 11400 raft_consensus.cc:2376] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14. Re-sending same reply.
I20260709 13:17:08.119514 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:08.117748 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:08.126101 11469 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.127447 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:08.129248 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.130231 11402 raft_consensus.cc:2368] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:17:08.131511 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.132450 11402 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:08.120669 16320 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:08.134251 16320 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.135421 16320 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:08.140410 16404 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.145605 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.146515 11399 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
I20260709 13:17:08.147415 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.116487 16142 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.147993 11399 raft_consensus.cc:2376] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14. Re-sending same reply.
I20260709 13:17:08.148883 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.150321 15630 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.103848 15705 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:08.152149 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 76605.9us
I20260709 13:17:08.155776 11470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2655) took 1040 ms. Trace:
I20260709 13:17:08.167431 11469 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:08.168759 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.170401 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.173421 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.173779 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:08.174224 11544 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:17:08.105909 16174 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.175325 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:08.176191 11543 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:08.177139 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.178313 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.179842 11469 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.179960 11400 raft_consensus.cc:2368] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:08.174393 11473 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:08.180002 11547 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:17:08.164870 16370 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:08.157423 11470 rpcz_store.cc:276] 0709 13:17:07.116545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.156054 (+ 39509us) service_pool.cc:224] Handling call
0709 13:17:07.156396 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:07.209541 (+ 53145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:07.209582 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:07.238226 (+ 28644us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.522584 (+284358us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:07.544220 (+ 21636us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.544444 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.544511 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.092523 (+548012us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:08.119851 (+ 27328us) raft_consensus.cc:1685] finished
0709 13:17:08.155510 (+ 35659us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.155699 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3108,"prepare.queue_time_us":23382,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":21804,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":16540,"spinlock_wait_cycles":3302144,"thread_start_us":246157,"threads_started":1,"wal-append.queue_time_us":305711}
I20260709 13:17:08.184976 11516 leader_election.cc:400] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
I20260709 13:17:08.163774 16399 consensus_queue.cc:1243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:08.188800 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.190042 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:08.190428 12084 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.191468 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.192170 12117 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:17:08.172798 16142 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:17:08.194343 16142 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:08.195400 15705 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.196664 15705 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.193610 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.199149 11551 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:08.194732 16142 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:08.201090 16412 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:08.201263 16399 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.203819 15630 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:17:08.151861 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08ebf4312af74188830a07342d0dc119"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:08.218732 11398 raft_consensus.cc:606] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.201743 16370 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:08.221580 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.222981 11551 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:08.223143 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.206003 15630 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:08.223856 11542 raft_consensus.cc:2393] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 10: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:08.228942 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.239287 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:08.224831 15630 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:08.241379 15630 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.224483 16399 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:08.242687 15630 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.252116 16174 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.253095 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.265642 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:08.266486 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:08.247061 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.254830 11545 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:08.268818 11550 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.254490 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.266832 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:08.255868 16370 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.261205 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.272305 11543 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:08.272451 11516 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.270128 11542 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
W20260709 13:17:08.248687 11477 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43866 (request call id 860) took 1580 ms (client timeout 1973 ms). Trace:
W20260709 13:17:08.273464 11477 rpcz_store.cc:269] 0709 13:17:06.668012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.685227 (+ 17215us) service_pool.cc:224] Handling call
0709 13:17:08.248482 (+1563255us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54004295d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:08.248590 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217804672}
I20260709 13:17:08.214324 16390 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:08.275269 16390 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.264552 11400 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:17:08.280542 11468 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:17:08.281908 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50227.4us
I20260709 13:17:08.284958 11469 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:08.282670 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:08.287698 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 48487.7us
I20260709 13:17:08.288661 11515 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:08.290014 11515 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:08.249459 16440 log.cc:927] Time spent T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:08.291853 16336 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:08.292598 16444 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.293318 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:08.242846 16175 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:08.325562 15080 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.287075 15705 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:08.328016 16336 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.335608 16119 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:08.355993 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:08.283947 16390 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:08.342090 16277 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:08.346743 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.371129 11549 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.318595 16311 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:08.338443 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:08.323666 16411 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.368907 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.366531 16119 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.354929 16425 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:08.340289 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1885) took 1112 ms. Trace:
I20260709 13:17:08.242825 16399 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.376159 15099 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:08.359445 16175 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:08.377828 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:17:08.366343 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 159888us
I20260709 13:17:08.391027 11543 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:08.399291 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 131501us
I20260709 13:17:08.326565 16174 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:17:08.403589 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:08.403995 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:08.404503 11395 raft_consensus.cc:606] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.405011 11395 raft_consensus.cc:614] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:08.405620 12086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.363862 16336 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:08.286818 16142 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.391793 16390 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.370869 16277 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:08.399809 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 168739us
I20260709 13:17:08.412672 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.374261 16311 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:08.400532 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.366441 16140 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:17:08.393110 15099 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.440102 15099 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:08.391513 11393 rpcz_store.cc:276] 0709 13:17:07.228130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.228306 (+   176us) service_pool.cc:224] Handling call
0709 13:17:08.340016 (+1111710us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400231c54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:08.340163 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239459584}
W20260709 13:17:08.416232 16445 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:17:08.411175 15705 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:08.415030 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.392004 16423 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:08.399459 16411 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.414004 16221 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:08.417209 16311 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.431862 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:08.420291 11399 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:08.409677 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.441627 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.412420 16412 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:08.409672 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.439663 16404 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.445559 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.470719 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.472280 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.459506 11395 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 8: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:08.464932 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.466127 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:08.476991 11393 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:08.458276 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:08.477829 11476 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:08.478899 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 188954us
I20260709 13:17:08.480213 16418 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:08.480826 16418 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.481353 16418 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:08.483246 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.484671 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.466892 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:08.486078 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 73141us
I20260709 13:17:08.489135 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:08.489916 11399 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.463200 15705 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:08.464221 16405 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:08.458559 16442 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:08.456550 16221 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.466460 16390 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:08.493637 12083 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.2:36161 timed out after 1.973s (SENT) [suppressed 36 similar messages]
W20260709 13:17:08.494417 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 230733us
I20260709 13:17:08.494124 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:08.443524 11549 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:08.485054 16142 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:08.368031 16446 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 13:17:08.489117 11442 connection.cc:256] RPC call timeout handler was delayed by -1.0199035000000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22890000000000002s for a response
I20260709 13:17:08.506073 15099 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.515767 11550 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:08.517145 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.508203 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:08.511821 11516 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:08.509162 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 105244us
I20260709 13:17:08.519717 11393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:08.521412 11395 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:08.524376 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:08.524787 11468 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:08.528357 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:08.529003 11549 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
W20260709 13:17:08.508203 16448 log.cc:927] Time spent T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:17:08.548177 15099 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.505005 16221 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.544107 16408 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.539654 16453 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.545149 11365 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:08.545338 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:08.546181 16388 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.205s	user 0.004s	sys 0.000s
I20260709 13:17:08.553412 11399 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:08.571977 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.577358 11469 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 5.
I20260709 13:17:08.581183 12122 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
W20260709 13:17:08.540431 16447 log.cc:927] Time spent T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:17:08.592911 11393 raft_consensus.cc:606] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:08.597775 11393 raft_consensus.cc:614] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:08.594179 15099 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:17:08.594110 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49054.3us
W20260709 13:17:08.603313 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 91581.7us
I20260709 13:17:08.464740 16175 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.606107 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.612519 11516 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.613287 15099 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:08.591152 16408 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.614181 11365 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:08.597234 16453 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.630900 15099 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.621680 16408 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.627864 16450 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:08.628718 16453 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:08.672005 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44509.5us
I20260709 13:17:08.638046 11395 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:17:08.649210 16454 log.cc:927] Time spent T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:17:08.651741 12119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 711) took 1953 ms. Trace:
I20260709 13:17:08.653434 16459 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:17:08.623570 15705 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.599666 16412 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:08.659264 16320 consensus_meta.cc:287] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.006s
I20260709 13:17:08.626915 16421 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:17:08.662914 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 87732.2us
I20260709 13:17:08.678124 12119 rpcz_store.cc:276] 0709 13:17:06.698333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.704649 (+  6316us) service_pool.cc:224] Handling call
0709 13:17:06.705081 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.768937 (+ 63856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:06.768981 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:06.787972 (+ 18991us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.187028 (+399056us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:07.187068 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.187137 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.545557 (+1358420us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:08.651240 (+105683us) raft_consensus.cc:1685] finished
0709 13:17:08.651267 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.651536 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6558,"raft.queue_time_us":11655,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":183,"spinlock_wait_cycles":34973440,"thread_start_us":398569,"threads_started":1,"wal-append.queue_time_us":532893}
I20260709 13:17:08.657311 11549 raft_consensus.cc:1240] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:08.674175 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 180552us
I20260709 13:17:08.693045 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:08.693379 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:08.698853 16455 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:08.699512 16455 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.702672 16455 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:08.703645 16421 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:08.705072 16421 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.707729 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.695474 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:08.724454 16456 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:08.725006 16456 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:08.725857 16456 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.678336 16399 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.729084 11551 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:08.744724 16459 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:08.747527 11550 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:08.742393 16277 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.751973 16277 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:08.762120 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 69160.3us
I20260709 13:17:08.763017 11516 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:08.763834 16450 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.765830 11516 leader_election.cc:400] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:17:08.766480 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:08.768745 16450 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:08.772691 16311 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.777706 11543 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:08.784039 11550 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:08.787330 15080 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.798589 15080 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.799074 15080 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.786787 11516 leader_election.cc:403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:17:08.789116 16277 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:17:08.793408 16416 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.918s	user 0.005s	sys 0.001s
W20260709 13:17:08.795703 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 97798.7us
I20260709 13:17:08.832270 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.833086 11543 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:08.835546 16460 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.839325 16439 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:08.842482 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.847944 15080 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.858171 16459 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.858814 16459 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.860875 16438 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:08.861299 16438 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:08.872381 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 106664us
I20260709 13:17:08.861277 15080 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.863022 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:08.874279 11516 leader_election.cc:400] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
I20260709 13:17:08.885499 11516 leader_election.cc:403] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:17:08.862353 16459 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.800321 16450 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.892414 11543 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:17:08.850046 16254 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.830928 16461 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.856459 16399 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10.
I20260709 13:17:08.810853 16462 consensus_queue.cc:1059] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:17:08.880409 16370 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.896406 16455 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14.
W20260709 13:17:08.915649 16119 consensus_meta.cc:287] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.000s
I20260709 13:17:08.916249 16119 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.862514 16438 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.900344 16254 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:08.932029 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 714) took 1375 ms. Trace:
I20260709 13:17:08.928920 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:08.933851 11543 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.927035 11394 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:08.932421 11396 rpcz_store.cc:276] 0709 13:17:07.564856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.565038 (+   182us) service_pool.cc:224] Handling call
0709 13:17:07.571357 (+  6319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:07.576656 (+  5299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:07.576691 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:07.580211 (+  3520us) log.cc:844] Serialized 52 byte log entry
0709 13:17:07.660073 (+ 79862us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18002c3b60. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:17:07.731039 (+ 70966us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:07.731081 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.731153 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.792706 (+1061553us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:08.930742 (+138036us) raft_consensus.cc:1685] finished
0709 13:17:08.930765 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.930910 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":380,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":15362688,"thread_start_us":70774,"threads_started":1,"wal-append.queue_time_us":224787}
I20260709 13:17:08.918951 16119 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:08.944701 16175 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.959403 16320 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:08.925872 16254 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:08.919652 16462 raft_consensus.cc:3055] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:08.946817 16390 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:08.993489 11543 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
W20260709 13:17:08.999228 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 90034.2us
I20260709 13:17:08.925475 16450 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:09.002187 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.006829 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.008309 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.009395 16411 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:09.010118 16411 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:09.010727 16411 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:09.002686 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.011788 11469 raft_consensus.cc:2376] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12. Re-sending same reply.
I20260709 13:17:09.012718 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:08.921844 16336 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.061910 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.074355 11477 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:17:09.063316 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47782us
I20260709 13:17:09.064023 12113 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:08.923141 16221 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:08.930899 16370 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:09.075755 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 74015us
W20260709 13:17:09.084333 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 76085.6us
I20260709 13:17:09.085207 11516 leader_election.cc:400] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:09.085526 11516 leader_election.cc:403] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:17:09.085848 16370 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.086480 16390 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Leader election lost for term 12. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:09.088366 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.075570 16423 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.000532 16462 raft_consensus.cc:740] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:09.094451 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.095273 11398 raft_consensus.cc:2368] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
W20260709 13:17:09.103434 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 116339us
I20260709 13:17:09.099222 11367 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:09.104429 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.108541 11402 raft_consensus.cc:2368] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 17. Current term is 18.
I20260709 13:17:09.108690 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.109542 11396 raft_consensus.cc:2376] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13. Re-sending same reply.
I20260709 13:17:09.096390 15705 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.127205 11516 leader_election.cc:400] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13.
I20260709 13:17:09.124007 16459 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.128453 16459 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:09.130478 16418 consensus_meta.cc:287] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.648s	user 0.002s	sys 0.008s
W20260709 13:17:09.134202 16142 consensus_meta.cc:287] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.616s	user 0.004s	sys 0.004s
I20260709 13:17:09.133975 16408 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.130623 16459 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:09.100452 16311 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.135089 11469 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:17:09.135494 11469 pending_rounds.cc:85] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 5
I20260709 13:17:09.113231 16421 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:09.155956 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.119789 16456 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.164019 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.130949 16418 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.157871 11367 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:09.096171 16423 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.135758 11469 pending_rounds.cc:107] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:17:09.175258 11469 raft_consensus.cc:2889] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:09.118197 16370 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:17:09.161387 16142 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.179709 16423 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.177222 16437 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:09.190903 16473 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:17:09.178387 16421 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:09.155383 15705 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.202919 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.187014 16140 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:17:09.207489 16476 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:17:09.185743 16456 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.199878 16418 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.200994 16459 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.220038 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.160236 16462 consensus_queue.cc:260] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:09.228425 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70641.9us
I20260709 13:17:09.230818 11442 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.201151 16142 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.226136 16451 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.221434 16140 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:09.234357 16456 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:09.228430 16459 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.243125 16336 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.244889 16174 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.281069 11442 leader_election.cc:400] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:17:09.259958 15705 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.285485 11442 leader_election.cc:403] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:09.276201 16462 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:09.288549 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:09.290774 11546 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:09.259691 16437 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:17:09.289610 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.293944 11549 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:09.291539 11551 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:09.297737 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.298357 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.291185 16451 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:17:09.299619 16451 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:09.299657 11442 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.302035 16456 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:09.303331 16453 consensus_meta.cc:287] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.005s
I20260709 13:17:09.303786 16453 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.307058 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.308291 16451 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.306694 16408 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.333499 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.339769 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:09.349143 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 110016us
I20260709 13:17:09.282945 16421 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.363117 11546 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:09.364285 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:09.367254 16336 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:09.367795 16336 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:09.368661 16336 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.370530 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.371057 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.371372 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.371685 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.372052 11399 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:09.372336 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.372742 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.372938 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.373494 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.373560 11399 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:09.374264 11398 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:17:09.374634 11398 pending_rounds.cc:85] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Aborting all ops after (but not including) 6
I20260709 13:17:09.375104 11398 pending_rounds.cc:107] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:17:09.375341 11398 raft_consensus.cc:2889] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:09.375707 11395 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:17:09.375797 16459 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.376034 16277 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.376730 11398 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:09.376530 16277 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.377137 11401 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:09.377971 16174 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.378749 16174 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:09.379421 16174 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.381029 11401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:09.381600 11401 raft_consensus.cc:606] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.378914 16277 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.374578 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.385010 11546 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:09.386806 11545 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:09.363799 16451 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:09.373601 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.388430 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:09.389153 11396 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:09.390122 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.390909 11469 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:09.390777 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.391458 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.392331 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.393564 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.394270 11472 raft_consensus.cc:2368] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:17:09.357579 16442 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:09.413033 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.415684 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.416640 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.418318 11471 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:17:09.309456 16140 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.424011 11469 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:09.415061 16175 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:09.416467 16439 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:09.407974 16442 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.408092 16451 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:09.357290 16469 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:17:09.427326 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.433854 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.434731 11469 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:09.430012 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:09.439693 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.440275 11469 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:09.423605 16142 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.429718 16439 consensus_queue.cc:1243] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:09.442730 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.432660 16442 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:09.451474 11516 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:09.448866 11549 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:09.451874 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.424716 16175 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.454214 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.473366 11395 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:09.455416 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.456415 11397 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:09.459852 16459 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:09.453222 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.477574 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.479276 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.455483 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.481463 11396 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:09.483649 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.479947 11397 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:09.441218 16451 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:09.487013 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.484608 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.489794 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.487865 11401 raft_consensus.cc:2376] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10. Re-sending same reply.
I20260709 13:17:09.489802 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.495074 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.495661 11395 raft_consensus.cc:2376] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15. Re-sending same reply.
I20260709 13:17:09.499562 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.477689 16461 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:09.478530 16459 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:09.505076 16221 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.508129 11544 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:09.509275 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.509655 11395 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:09.505769 11402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 2158) took 1034 ms. Trace:
I20260709 13:17:09.519668 11402 rpcz_store.cc:276] 0709 13:17:08.471844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.471964 (+   120us) service_pool.cc:224] Handling call
0709 13:17:09.505593 (+1033629us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b540030aa54. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:09.505714 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459580160}
I20260709 13:17:09.524654 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:09.494719 16411 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:09.533143 11393 raft_consensus.cc:2468] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:09.525794 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.528311 16451 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.480062 16450 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:17:09.479454 16412 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:17:09.528345 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.531148 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.533859 11396 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:09.553666 11402 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:09.536404 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.554877 11395 raft_consensus.cc:2410] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:09.534595 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:09.493085 16399 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:17:09.547163 16405 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.502282 16461 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.521325 16459 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:09.558120 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:09.565395 11368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.568279 16453 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.526595 16142 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.532625 16411 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:09.579219 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44731.8us
I20260709 13:17:09.579444 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.555122 16174 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:09.585829 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.588891 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.586762 16174 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.584491 16142 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.590838 16174 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:09.592449 16142 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.582893 16325 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.593251 16325 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.555866 16175 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.574107 16461 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:09.581918 16437 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:09.584856 16411 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:09.597265 16142 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:09.620658 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:09.620100 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.621475 11551 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:09.608758 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:09.622279 11545 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:09.623263 15807 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:09.617651 15630 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.625427 15630 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.627672 15630 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.581585 16405 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.628914 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.634486 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:09.603464 16437 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.640937 16174 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.607200 16430 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.643306 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.644972 16430 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.647032 11397 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:09.647287 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:09.658223 12115 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 19: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:09.662773 11545 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 6.
I20260709 13:17:09.652383 16420 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:09.693778 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:17:09.715170 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 114377us
I20260709 13:17:09.716398 16142 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.717046 16142 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.722669 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.723992 16142 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.609760 16418 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:17:09.667303 16437 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:09.743745 16430 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.656543 16405 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:09.778538 16482 log.cc:927] Time spent T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.505s	user 0.000s	sys 0.002s
I20260709 13:17:09.785566 11476 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:09.787478 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.788070 16438 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:09.788444 11469 raft_consensus.cc:2368] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:17:09.789956 16446 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:09.791824 16446 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:09.793119 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.793394 16446 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:09.793943 11476 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:09.796631 16370 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.797257 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.797346 16370 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.800475 11468 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:17:09.803813 11442 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:09.803895 11549 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:09.808000 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:09.808755 11550 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 6: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:09.809814 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:09.810528 11546 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 19: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:09.811658 11514 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.818259 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.818936 16422 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:09.819397 16422 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:09.815713 16370 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.820214 16422 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.821482 16422 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:09.821591 16370 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.822760 16370 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.826403 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.826747 16370 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.827541 11472 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:09.828099 16370 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:09.829093 16370 raft_consensus.cc:697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:09.830106 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.831481 11476 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:09.830744 16370 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.831864 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.832475 11470 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:09.832522 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:09.833173 11476 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:09.834307 11476 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:09.830751 11475 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:17:09.838456 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:09.839114 11472 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:09.846575 11296 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 1 to 8, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 8 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:09.849392 11470 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:09.851087 16455 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.853605 16455 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:09.854126 16455 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:09.855787 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44211.1us
I20260709 13:17:09.858626 16390 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:09.861876 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.863327 16390 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:09.865006 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:09.866325 16390 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.879165 16390 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:09.689193 16442 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.885197 16174 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.885411 15080 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.885984 15080 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.886590 16442 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.889714 16423 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:09.890312 16423 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.892920 16423 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:09.893976 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:09.895530 11400 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:09.897733 11546 pending_rounds.cc:85] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 6
I20260709 13:17:09.898131 11546 pending_rounds.cc:107] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:17:09.898430 11546 raft_consensus.cc:2889] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:09.889488 16459 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:09.905004 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:09.906487 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 278192us
I20260709 13:17:09.877238 11472 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:09.912796 16412 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:09.915174 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:09.917856 15080 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:09.918097 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:09.927510 16501 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:09.929358 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.931514 16503 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.958518 11514 leader_election.cc:400] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:17:09.958864 11514 leader_election.cc:403] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:17:09.970211 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62884.7us
I20260709 13:17:09.975235 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:09.890750 16411 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:09.997534 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.002812 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.003487 11544 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:09.985070 16175 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:10.003916 11365 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:10.004871 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:10.005316 16336 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:10.007143 16423 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:10.006503 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.010092 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:10.011083 12119 raft_consensus.cc:2368] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:17:09.976261 16430 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:10.017670 15099 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:17:10.025604 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.026278 11470 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:09.978065 16325 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.027650 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:10.029538 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:09.990300 16450 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 5. Current term is 6.
I20260709 13:17:10.011850 16462 raft_consensus.cc:993] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:10.034574 11545 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:10.035579 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.035707 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.038314 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.009934 16451 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:10.039866 15080 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:10.040267 15099 raft_consensus.cc:1081] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:10.047295 15080 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.039105 11544 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:09.985531 16399 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:17:09.985352 16418 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:17:10.016724 16469 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.008123 16235 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:17:10.029423 15630 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:10.051745 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.019543 16311 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:10.053305 12115 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:10.053479 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.053650 11544 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:10.007622 16442 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:17:10.049997 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:10.058864 11469 raft_consensus.cc:2393] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 14: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:10.065980 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.067261 12112 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 6: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:10.066428 16459 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.046502 16439 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.076673 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:09.989856 16503 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.059584 16469 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.039561 16119 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:17:10.058324 15630 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.109202 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.034723 16430 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.109182 12122 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:17:10.087020 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.148841 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.166620 11394 raft_consensus.cc:1240] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:17:10.171541 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:10.172020 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 154767us
I20260709 13:17:10.078269 16439 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.081677 16459 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:10.198738 11516 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:10.058033 16451 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.199892 11441 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.201828 16501 raft_consensus.cc:2804] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:10.114145 16469 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.156728 15630 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.209723 11441 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.068427 16311 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.197897 16439 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.202911 16175 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:17:10.247452 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.256088 16370 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:10.254915 16451 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.244824 16509 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:10.252758 16399 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.267949 16321 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.252084 16503 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:10.275976 16517 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:17:10.244629 15807 consensus_queue.cc:1059] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:17:10.286167 16502 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.520s	user 0.003s	sys 0.000s
W20260709 13:17:10.284864 16488 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.915s	user 0.001s	sys 0.012s
I20260709 13:17:10.278971 15630 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.243643 16513 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 13:17:10.296207 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:17:10.268265 16476 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:17:10.310861 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 112287us
I20260709 13:17:10.330466 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.308295 16520 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:10.312439 16399 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.336162 16399 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.312490 16411 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:10.341719 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 99461.4us
I20260709 13:17:10.343884 16501 raft_consensus.cc:697] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:10.345774 11394 raft_consensus.cc:1240] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:10.350005 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.356707 16336 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:10.357276 16336 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:10.357746 16508 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:17:10.362681 11402 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:10.368261 11477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43866 (request call id 878) took 1369 ms. Trace:
W20260709 13:17:10.369395 16506 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:17:10.364284 16459 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.370194 16459 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.362771 16521 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:17:10.371992 16504 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
I20260709 13:17:10.362257 16526 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:10.368783 11477 rpcz_store.cc:276] 0709 13:17:08.998729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.088572 (+ 89843us) service_pool.cc:224] Handling call
0709 13:17:09.088891 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.089264 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:09.180814 (+ 91550us) log.cc:844] Serialized 14 byte log entry
0709 13:17:09.280843 (+100029us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.285030 (+  4187us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.285131 (+   101us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:09.285152 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.285197 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.284243 (+999046us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:10.367953 (+ 83710us) raft_consensus.cc:1685] finished
0709 13:17:10.368051 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.368195 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1499,"prepare.queue_time_us":143331,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2455,"raft.queue_time_us":1157,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":10155,"spinlock_wait_cycles":85949696,"thread_start_us":155734,"threads_started":2,"wal-append.queue_time_us":161945}
I20260709 13:17:10.374509 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.381451 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.391561 16423 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:10.391844 15630 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.392025 16321 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:10.392619 16321 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.395814 11400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41730 (request call id 1940) took 1025 ms. Trace:
I20260709 13:17:10.396206 11400 rpcz_store.cc:276] 0709 13:17:09.370719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.372538 (+  1819us) service_pool.cc:224] Handling call
0709 13:17:10.395621 (+1023083us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540019a354. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:10.395755 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200555392}
I20260709 13:17:10.400775 16321 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.403394 16321 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.409664 16408 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:10.410290 16408 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:10.411641 16461 consensus_meta.cc:287] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.809s	user 0.010s	sys 0.001s
I20260709 13:17:10.412696 16408 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.409211 16336 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:10.413964 16438 consensus_meta.cc:287] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 13:17:10.414834 11398 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:10.415395 16336 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.412215 16461 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.341544 16520 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.417307 16430 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.429749 12123 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:10.430168 16439 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.431074 16461 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.432245 16439 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.434321 16501 consensus_queue.cc:237] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.387063 16521 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:10.436926 11395 raft_consensus.cc:1240] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:10.420708 16408 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:17:10.439886 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 65455.3us
I20260709 13:17:10.444214 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:10.444561 16520 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.448174 16459 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.451339 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.452105 12119 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:10.453006 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:17:10.456063 16492 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.074s	user 0.000s	sys 0.005s
I20260709 13:17:10.457319 12113 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 721) took 1209 ms. Trace:
I20260709 13:17:10.457661 12113 rpcz_store.cc:276] 0709 13:17:09.261645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.272646 (+ 11001us) service_pool.cc:224] Handling call
0709 13:17:09.282315 (+  9669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.286870 (+  4555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.286892 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.287309 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.357656 (+ 70347us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:09.357715 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.357826 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.434827 (+1077001us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:10.457056 (+ 22229us) raft_consensus.cc:1685] finished
0709 13:17:10.457077 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.457265 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8941,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2588,"spinlock_wait_cycles":53001472,"thread_start_us":45396,"threads_started":1,"wal-append.queue_time_us":60396}
I20260709 13:17:10.459865 16521 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:10.461038 16511 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.388s	user 0.002s	sys 0.000s
I20260709 13:17:10.463111 16469 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.465188 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.465684 16509 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:10.466105 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:10.467047 16523 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:17:10.453006 16455 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.483369 11471 raft_consensus.cc:2393] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 14: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:10.480295 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:10.473233 16437 consensus_meta.cc:287] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.740s	user 0.013s	sys 0.000s
I20260709 13:17:10.492408 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.496937 16469 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.498843 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:10.499692 12123 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:17:10.503809 16486 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 1.230s	user 0.006s	sys 0.000s
I20260709 13:17:10.505446 11473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 2699) took 1541 ms. Trace:
I20260709 13:17:10.506080 11473 rpcz_store.cc:276] 0709 13:17:08.964496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.964614 (+   118us) service_pool.cc:224] Handling call
0709 13:17:09.004822 (+ 40208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.090514 (+ 85692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.090544 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.116926 (+ 26382us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.200766 (+ 83840us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:09.200804 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.165935 (+965131us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b540028b754. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:10.165986 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.503172 (+337186us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:10.505209 (+  2037us) raft_consensus.cc:1685] finished
0709 13:17:10.505232 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.505388 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":192040960,"thread_start_us":74495,"threads_started":1,"wal-append.queue_time_us":91128}
I20260709 13:17:10.493991 16176 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:10.511556 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.514575 16450 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:10.515208 16450 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.517493 16450 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.518051 16450 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.518194 16311 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.518539 16450 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.521112 16450 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:10.521175 15080 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.521391 16509 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:10.576717 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.499887 16437 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.526495 11398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41728 (request call id 2163) took 1073 ms. Trace:
I20260709 13:17:10.527556 11473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:10.579429 11473 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.580114 11473 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:10.530098 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.531301 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.532398 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.533825 11550 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:17:10.534835 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.535815 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.541252 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.541985 11549 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:10.543720 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.548542 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.550392 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.554133 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.582947 11365 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.597280 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:10.554361 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.555156 11515 leader_election.cc:400] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:17:10.601758 11515 leader_election.cc:403] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:17:10.602767 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 47762.2us
I20260709 13:17:10.603757 11515 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.568507 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.605633 11544 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:10.608788 11476 raft_consensus.cc:2393] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:10.562427 16412 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:10.495424 16521 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:10.578308 11398 rpcz_store.cc:276] 0709 13:17:09.452875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.452989 (+   114us) service_pool.cc:224] Handling call
0709 13:17:10.526257 (+1073268us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b5400126bd4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da991d2fa 00007f8dae8deb17 00007f8da7e596d4 00007f8da7e59662 00007f8da7e59591 00007f8da7e5950c 00007f8da7e587c2 00007f8da7c5448c 00007f8da7c538c6 00007f8da7c5326f
0709 13:17:10.526414 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161642880}
I20260709 13:17:10.611807 11470 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:10.600143 11469 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:10.622414 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.562358 16459 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:10.628441 11470 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.629683 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.647416 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.616964 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.617908 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:10.618770 11474 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:17:10.571110 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.595618 11547 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:10.691342 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.772823 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:10.773109 11469 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:10.773320 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.632952 16439 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:10.711293 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 740) took 1125 ms. Trace:
I20260709 13:17:10.774688 16540 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:10.775311 16540 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.623436 16509 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.778065 16540 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:10.778146 16509 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:10.727319 16507 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.504s	user 0.002s	sys 0.005s
I20260709 13:17:10.778893 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:10.780360 11401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41728 (request call id 2179) took 1243 ms. Trace:
I20260709 13:17:10.613586 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.614534 16501 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:17:10.738646 11516 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:10.781419 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.782019 16390 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:10.782639 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.782827 16390 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:10.782924 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:10.783713 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 170234us
I20260709 13:17:10.784745 16529 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:10.784924 16399 consensus_queue.cc:1059] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:10.738762 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.785545 16529 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:10.787720 11469 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:10.787868 11468 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:10.573094 16254 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:10.742120 11398 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:17:10.790406 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.791216 11398 raft_consensus.cc:2368] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
I20260709 13:17:10.742561 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:10.743197 16420 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:10.802335 16420 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:10.743434 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.802721 16420 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:10.786929 16529 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:10.803472 16529 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:10.804049 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:10.804594 16537 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:17:10.804330 16529 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.805300 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:10.746925 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.805773 16529 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:10.806656 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.747232 12115 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:17:10.802645 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.748066 16405 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:10.749006 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.749749 16311 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.828930 11542 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:10.751045 11396 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:17:10.751058 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.831144 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.831796 11402 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:10.572907 16140 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.840245 16140 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.840795 11469 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:10.842571 16140 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.843503 16526 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:10.843699 11396 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:10.845731 11397 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:10.758910 16437 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:10.624348 16412 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:10.733182 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 178666us
I20260709 13:17:10.775063 11395 rpcz_store.cc:276] 0709 13:17:09.585701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.604496 (+ 18795us) service_pool.cc:224] Handling call
0709 13:17:09.618560 (+ 14064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:10.440831 (+822271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.710674 (+269843us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54003a70d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9919e01 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:10.710941 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881587584}
I20260709 13:17:10.620939 16455 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:10.850016 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.792006 11471 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.791306 11470 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:10.780880 11401 rpcz_store.cc:276] 0709 13:17:09.537365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.571133 (+ 33768us) service_pool.cc:224] Handling call
0709 13:17:09.571600 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.572084 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.572150 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.586070 (+ 13920us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.941979 (+355909us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:09.942021 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.942085 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.542433 (+600348us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:10.779967 (+237534us) raft_consensus.cc:1685] finished
0709 13:17:10.780000 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.780272 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":917,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7810,"spinlock_wait_cycles":371205376,"thread_start_us":355617,"threads_started":1,"wal-append.queue_time_us":464305}
I20260709 13:17:10.801553 11545 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:10.801692 16390 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:10.797358 16453 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:10.745210 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.809728 12085 leader_election.cc:400] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
W20260709 13:17:10.811877 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 74486.3us
I20260709 13:17:10.864944 12085 leader_election.cc:403] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:10.762161 16459 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.869554 15080 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:10.869796 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 60318.2us
I20260709 13:17:10.870630 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.800931 11473 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.834229 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:10.836078 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:10.796871 16510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.003s	sys 0.008s
I20260709 13:17:10.837529 11472 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:10.838660 12117 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:10.838745 16405 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:10.839543 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:10.839753 11402 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:10.624373 12123 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:10.872617 11469 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:10.874714 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:10.870958 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 125833us
I20260709 13:17:10.822783 16119 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:10.905967 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 69990.6us
I20260709 13:17:10.913110 11471 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:10.913811 11473 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:10.916460 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66546.5us
I20260709 13:17:10.917526 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.803957 16420 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:10.900638 16510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.003s	sys 0.008s
I20260709 13:17:10.859026 16140 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.869555 16540 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:10.904136 16488 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:10.905524 16430 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:10.899642 16505 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
I20260709 13:17:10.844594 16254 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.845866 16311 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:10.900033 16405 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:17:10.920030 16538 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:10.948410 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:10.938421 16492 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:17:10.956390 11398 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:10.916034 16526 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:10.921303 16520 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.921662 16450 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:10.915181 16421 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:10.986188 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 68744.6us
I20260709 13:17:10.938225 16460 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.030903 11544 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:11.058038 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 71356.3us
I20260709 13:17:11.031720 12113 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:10.953449 16510 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:10.949338 16411 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:17:11.047472 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 172837us
I20260709 13:17:11.065711 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:10.915069 16119 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:10.944492 16238 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:17:10.981256 16420 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:11.064739 11367 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:10.952617 16487 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
W20260709 13:17:11.087988 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 243990us
I20260709 13:17:10.949203 16140 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.083086 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.056710 16450 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.099315 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:11.132149 11544 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:17:11.044268 16176 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
W20260709 13:17:11.147858 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 48619.9us
I20260709 13:17:11.085589 16238 consensus_queue.cc:1243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 9e9c2603c0bd4d25b713649b3890c46b log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:17:11.155825 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 98147.7us
I20260709 13:17:11.028477 16390 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.099404 16411 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.123133 16501 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.099619 16469 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:11.081960 16421 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.126377 15807 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:11.085407 16503 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:11.083427 16460 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.129585 16450 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:11.147341 16510 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.135991 16336 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:11.155288 16321 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:11.060721 16520 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.123234 16498 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 13:17:11.178889 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50599.9us
I20260709 13:17:11.214627 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:11.231524 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 100552us
I20260709 13:17:11.148128 16405 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.221449 16254 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:11.223923 16412 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:11.172133 16411 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.213807 16469 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.242602 16421 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:11.260075 16541 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:11.262499 16119 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:11.263216 16460 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:11.263382 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 48901.6us
I20260709 13:17:11.239789 16399 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:11.234092 11365 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:17:11.285182 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51183.6us
I20260709 13:17:11.269729 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:11.242522 16530 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:11.274101 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 114632us
I20260709 13:17:11.177790 16501 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.293731 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.295169 11516 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:11.296159 16460 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:11.299104 16460 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:17:11.270594 16542 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:17:11.248459 16321 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.235764 16510 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.235737 16336 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.292670 16174 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.284142 16311 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.297513 16119 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:11.284425 16399 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:11.310949 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.299726 16460 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:11.304366 16529 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:11.270601 16412 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:17:11.353578 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:11.355032 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:11.359001 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:11.319613 16540 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:11.305032 16238 consensus_queue.cc:1059] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:17:11.348435 16311 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.352743 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:11.310272 16235 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:17:11.378377 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 84715.6us
I20260709 13:17:11.344511 16555 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.346244 16119 raft_consensus.cc:697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.319211 16176 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:17:11.319408 16174 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.359614 16554 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:11.418156 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 65491.3us
I20260709 13:17:11.368618 16140 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:11.350864 16469 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.432369 16175 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:11.432941 16175 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:11.433923 16175 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.436364 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:11.437153 12117 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:11.438905 16336 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:11.443931 16536 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.692s	user 0.009s	sys 0.000s
I20260709 13:17:11.445180 16174 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:11.446869 16119 consensus_queue.cc:237] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.450910 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 1949) took 1537 ms. Trace:
I20260709 13:17:11.416540 11442 leader_election.cc:400] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 17. Current term is 18.
I20260709 13:17:11.360945 16529 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:17:11.415086 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 137523us
I20260709 13:17:11.454470 11393 rpcz_store.cc:276] 0709 13:17:09.913210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.916952 (+  3742us) service_pool.cc:224] Handling call
0709 13:17:09.917380 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.919896 (+  2516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.919942 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:10.474630 (+554688us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b2c00000658. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da8212a11 00007f8da8222372 00007f8da9954a01 00007f8da991943a 00007f8da991883e 00007f8dae8dd419 00007f8da7e54bd4 00007f8da7e54b62 00007f8da7e54a91 00007f8da7e54a0c 00007f8da7e53cc2
0709 13:17:10.475104 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:17:10.628573 (+153469us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:10.628625 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.628700 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.443459 (+814759us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b2400137108. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da66b29c4 00007f8da66b0acb 00007f8da686bdc9 00007f8da686ba55 00007f8da686b09d 00007f8da949d10e 00007f8da94b488c 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221
0709 13:17:11.450655 (+  7196us) raft_consensus.cc:1685] finished
0709 13:17:11.450681 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.450848 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2056,"raft.queue_time_us":2574,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":73056512,"thread_start_us":152580,"threads_started":1,"wal-append.queue_time_us":275943}
W20260709 13:17:11.463791 16559 log.cc:927] Time spent T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:17:11.464224 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:11.465400 16547 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:17:11.464816 16529 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.432360 16501 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:11.467695 16572 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:17:11.468165 16439 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:11.468767 16439 consensus_queue.cc:1243] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:17:11.468849 16175 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:11.472441 12112 pending_rounds.cc:85] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 11
I20260709 13:17:11.472827 12112 pending_rounds.cc:107] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:17:11.473085 12112 raft_consensus.cc:2889] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:11.404747 16555 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.421147 16421 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:11.423820 16390 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:11.424793 16520 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:11.464811 16553 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:17:11.374085 16399 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.402805 16311 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.438931 16459 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.367726 16321 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.419659 16540 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:17:11.487900 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 177018us
I20260709 13:17:11.507781 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:11.537585 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 75787.9us
I20260709 13:17:11.538416 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.547241 11393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:11.547976 11393 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:11.548955 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.551405 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.500311 16555 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.556707 16510 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:11.557422 11544 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:17:11.562781 16411 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.427490 16554 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.573572 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:11.575309 11545 pending_rounds.cc:85] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 6
I20260709 13:17:11.585088 11545 pending_rounds.cc:107] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:17:11.576960 16450 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.585386 11545 raft_consensus.cc:2889] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:11.578253 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.587200 11542 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:11.581703 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.588028 11393 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:11.588382 11548 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:17:11.589509 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51143.7us
I20260709 13:17:11.582734 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.591356 11550 raft_consensus.cc:2410] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:11.592389 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.593128 11393 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:11.553602 16459 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.593586 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.597682 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.553375 16336 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:17:11.567379 16553 log.cc:927] Time spent T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:17:11.556074 16399 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:11.601374 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.593446 11367 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:11.599287 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.588826 16510 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:11.597193 16430 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.597103 16520 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.598887 16509 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:11.619525 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.621095 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.623872 16530 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:11.626823 16311 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.603089 16459 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.623247 16510 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:11.621699 16336 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:11.612315 16576 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:11.617532 16430 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.639154 16540 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:11.640710 16582 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:11.641776 16460 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.644130 11547 raft_consensus.cc:2393] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:11.641217 16450 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.656786 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.657651 11397 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:17:11.661489 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:11.664921 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 353912us
I20260709 13:17:11.618458 16520 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.655607 16540 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:17:11.704553 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:11.688313 16175 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:17:11.706887 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44205.5us
I20260709 13:17:11.716153 11470 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:17:11.717694 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:11.722877 11365 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.725279 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:11.706725 16430 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.692229 16576 consensus_queue.cc:1059] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:17:11.720549 16526 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:11.705555 16587 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:11.733001 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:11.733983 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:11.749174 11545 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:11.737082 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.736560 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:11.738132 11516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.738221 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:11.753628 11550 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:11.738632 16311 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:11.754452 16311 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.756806 16311 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.757045 16411 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.738741 16572 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.757695 16411 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.758061 16572 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:11.760375 16520 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.743242 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.743482 16555 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.753123 16176 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:11.762064 16555 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.745185 16586 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:17:11.772517 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.773824 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:11.735275 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.774686 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:11.775094 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.775926 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:11.776185 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.739746 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:11.766939 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.769207 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.729140 16509 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:11.778446 11393 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:11.779104 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.779307 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:11.779791 11393 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:11.771342 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.767302 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:11.770542 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.783305 11397 raft_consensus.cc:2393] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:11.783361 11398 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:11.727308 15705 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:17:11.782341 11400 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:11.783842 11398 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.727618 16460 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.748208 16526 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.748510 16587 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.791397 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:11.792130 11545 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:11.793406 11550 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:11.798095 16455 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.796959 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:11.743670 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.721036 16175 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:11.802093 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 58561.9us
I20260709 13:17:11.810679 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:11.743696 16529 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.742074 16577 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.785310 16520 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:11.742383 16174 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:11.795740 16587 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:11.809191 11398 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.827510 16176 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.828437 16176 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.830446 16174 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.809064 16540 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:11.831465 15705 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:11.831873 15705 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:11.832862 16174 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:11.832614 15705 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.821213 16509 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:11.833798 15705 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:11.835264 16405 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:11.843957 16569 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
W20260709 13:17:11.825964 15080 consensus_queue.cc:1175] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34fe1737ebb840fb9746980f649cc0a6" responder_term: 13 status { last_received { term: 12 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:11.799731 16461 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:11.795518 16526 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:11.854580 11398 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:11.855611 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 136040us
I20260709 13:17:11.809229 16412 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:11.857749 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.860093 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.863770 11397 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:11.874414 11398 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:11.826498 16520 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:11.875811 11393 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
W20260709 13:17:11.859418 16561 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.598s	user 0.000s	sys 0.003s
I20260709 13:17:11.886520 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.798735 16455 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.791101 16311 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:17:11.857234 16555 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:11.826757 16529 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.921339 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.886425 16520 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:11.932070 16529 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:11.933701 16593 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:17:11.935410 16596 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:17:11.900249 16582 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.852224 16577 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.945256 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.922133 16399 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:11.963634 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 150293us
W20260709 13:17:11.970949 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.979908 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:11.985395 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:11.971822 16469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.010s	sys 0.005s
I20260709 13:17:11.992103 16577 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:11.962810 16582 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.998494 15080 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:11.999114 15080 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:11.982851 16571 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:11.997741 16469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.010s	sys 0.005s
I20260709 13:17:12.001358 16411 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.024082 15080 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:12.033950 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54048.3us
I20260709 13:17:12.017745 16571 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.109633 16603 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:17:12.110571 16571 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:12.111681 16592 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:17:12.042613 16437 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 13:17:12.116281 16599 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:17:12.068490 16577 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:17:12.126675 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.148649 16399 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.149432 16175 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.150297 16399 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.153668 16399 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:12.154107 16459 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.159336 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.160039 11551 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:17:12.160796 16455 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:12.111716 16608 log.cc:927] Time spent T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:17:12.171728 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:12.120507 16501 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.174105 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.182788 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.183509 11393 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:17:12.173115 16455 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:12.199240 16455 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.186036 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.177922 16501 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.202376 16455 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:12.193099 11297 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 245) took 34547 ms (client timeout 15000 ms). Trace:
I20260709 13:17:12.194877 16405 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:17:12.206126 16594 log.cc:927] Time spent T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.408s	user 0.001s	sys 0.001s
I20260709 13:17:12.206430 16405 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.206601 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.206880 16405 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:12.195567 12113 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:12.208882 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.211093 16586 raft_consensus.cc:3055] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:12.211491 16586 raft_consensus.cc:740] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:12.212236 16586 consensus_queue.cc:260] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.213409 16586 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:12.217083 16520 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.218164 11545 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:12.219300 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:12.183079 16617 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:17:12.222395 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:12.224674 11393 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:12.229252 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.228168 16587 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:12.230353 16551 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.968s	user 0.005s	sys 0.000s
I20260709 13:17:12.230818 16501 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.231578 16587 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.220856 16520 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.234001 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.234738 11393 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:17:12.241586 11473 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: [11.6-13.7]
W20260709 13:17:12.204815 11297 rpcz_store.cc:269] 0709 13:16:37.645600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.646047 (+   447us) service_pool.cc:224] Handling call
0709 13:17:12.192875 (+34546828us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:03.256818 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:03.483756 (+226938us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:03.490786 (+  7030us) write_op.cc:435] Acquired schema lock
0709 13:17:03.494624 (+  3838us) tablet.cc:662] Decoding operations
0709 13:17:03.942589 (+447965us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:03.959749 (+ 17160us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:03.974656 (+ 14907us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:17:04.620981 (+646325us) tablet.cc:701] Row locks acquired
0709 13:17:04.621003 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:17:04.621156 (+   153us) write_op.cc:270] Start()
0709 13:17:04.621286 (+   130us) write_op.cc:276] Timestamp: P: 1783603024621106 usec, L: 0
0709 13:17:04.621311 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:04.622004 (+   693us) log.cc:844] Serialized 6957 byte log entry
0709 13:17:05.269466 (+647462us) op_driver.cc:464] REPLICATION: finished
0709 13:17:05.598798 (+329332us) write_op.cc:301] APPLY: starting
0709 13:17:05.598885 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:05.873717 (+274832us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:05.873827 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:06.422367 (+548540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:06.422969 (+   602us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:17:06.423105 (+   136us) write_op.cc:312] APPLY: finished
0709 13:17:06.639606 (+216501us) log.cc:844] Serialized 164 byte log entry
0709 13:17:07.053058 (+413452us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:07.169083 (+116025us) write_op.cc:454] Released schema lock
0709 13:17:07.192785 (+ 23702us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:07.255220 (+ 62435us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:07.255264 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76809,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":329081,"apply.run_cpu_time_us":88336,"apply.run_wall_time_us":1645698,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":143440,"prepare.run_cpu_time_us":15915,"prepare.run_wall_time_us":1519463,"raft.queue_time_us":205223,"raft.run_cpu_time_us":119575,"raft.run_wall_time_us":315665,"replication_time_us":647999,"thread_start_us":990546,"threads_started":5,"wal-append.queue_time_us":582399,"wal-append.run_cpu_time_us":23881,"wal-append.run_wall_time_us":2724866}]]}
I20260709 13:17:12.251932 16430 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:12.252559 16430 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.259558 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:12.261773 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:12.261929 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:12.262492 16565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.000s	sys 0.006s
I20260709 13:17:12.268570 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.273104 16582 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.284005 16405 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.284848 16530 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:12.324471 16530 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.326599 16530 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.327862 16423 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.328289 16605 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:12.328433 16423 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.328855 16605 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:12.329979 16560 log.cc:927] Time spent T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 1.067s	user 0.004s	sys 0.004s
I20260709 13:17:12.330641 16423 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:12.323539 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:12.285028 16565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.000s	sys 0.007s
I20260709 13:17:12.285626 16461 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:12.286154 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.288691 11471 raft_consensus.cc:3060] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:12.289062 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.290313 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.290941 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.295178 16430 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.233969 16526 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.295324 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.339568 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:12.339779 16546 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 1.177s	user 0.000s	sys 0.006s
I20260709 13:17:12.340662 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:12.285575 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.342511 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.343168 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.343797 11475 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:17:12.344199 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.344579 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.296252 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.314913 16554 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:12.316181 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 50586.6us
I20260709 13:17:12.332158 11545 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:12.347074 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.347671 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.348104 11476 raft_consensus.cc:2410] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:17:12.348816 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.349145 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:12.350181 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:12.333016 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:12.352738 12118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55074 (request call id 732) took 1564 ms. Trace:
I20260709 13:17:12.358597 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:12.367866 11516 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:12.369177 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.353224 12118 rpcz_store.cc:276] 0709 13:17:10.788535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.788641 (+   106us) service_pool.cc:224] Handling call
0709 13:17:10.788936 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:10.789249 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:10.789273 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:10.789839 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:17:10.935633 (+145794us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:10.935678 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.935752 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.339099 (+1403347us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b4400022b4c. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da6752c85 00007f8da67536a2 00007f8da94c51db 00007f8da94c0aaf 00007f8da94b486e 00007f8da94b3bb6 00007f8da94b37d8 00007f8da94bd221 00007f8da94bd1d9 00007f8da94bd169
0709 13:17:12.341548 (+  2449us) raft_consensus.cc:1685] finished
0709 13:17:12.341571 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.352645 (+ 11074us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":7654784,"thread_start_us":145390,"threads_started":1,"wal-append.queue_time_us":309271}
I20260709 13:17:12.372943 16438 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:12.365556 16501 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:12.373399 16438 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:12.362613 16554 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:12.374248 16438 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.375358 16438 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:12.376351 16577 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:12.376927 16577 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.335114 16520 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:12.383315 16577 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.387301 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.387952 11367 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:12.388612 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.388629 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.390100 16498 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:12.391734 16442 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:12.390053 11548 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:12.393014 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:12.393589 11544 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:12.392740 16421 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:12.395267 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.396534 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:12.406910 11471 raft_consensus.cc:1275] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:12.408090 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:12.435654 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49882.7us
I20260709 13:17:12.408821 16421 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:12.388630 16572 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:12.409607 11547 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:12.409706 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:12.411101 16501 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:12.414180 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.370635 16461 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:12.423806 11544 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:12.424453 11468 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:12.353358 16605 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:12.437903 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:12.439538 11475 raft_consensus.cc:2410] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:17:12.363982 16530 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:12.429693 16576 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:12.367316 16505 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:12.457559 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:12.459859 11470 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:12.461165 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.461200 11468 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:12.465561 11365 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:12.436211 16554 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:12.490109 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:12.468472 15080 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:12.454048 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.477599 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.492394 16336 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:17:12.507050 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 111679us
I20260709 13:17:12.512334 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.516242 11471 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:12.494560 16554 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:12.464031 16459 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:17:12.503283 16572 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:12.522321 12083 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:12.543068 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.509470 16336 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:12.475255 16530 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.496405 16576 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.520991 16580 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:12.495977 16505 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.539857 16572 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.536247 16554 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:12.568415 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.553114 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.585188 11473 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:12.552311 16336 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:12.604928 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.605856 11477 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 8.
I20260709 13:17:12.594380 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:17:12.596043 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 130609us
I20260709 13:17:12.607136 11473 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:12.608685 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 86496.8us
I20260709 13:17:12.611184 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:12.584180 16572 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:12.628553 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 131266us
W20260709 13:17:12.636588 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 194298us
I20260709 13:17:12.602934 16405 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.640041 11474 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:12.640705 11398 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:17:12.649142 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.649559 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.650307 11548 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:12.618868 16550 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:12.643478 11550 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:12.653200 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:12.644236 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.643283 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.654145 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.642213 11394 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:12.642596 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.651392 11547 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:17:12.551352 16501 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.654654 11397 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:12.657145 11395 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:12.657181 11400 raft_consensus.cc:2393] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:12.635929 16526 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:12.678066 11473 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:12.683740 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:12.650570 16459 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:17:12.692575 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44684.9us
W20260709 13:17:12.697811 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 65178us
I20260709 13:17:12.704147 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.704866 11543 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:12.710695 11477 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:12.715145 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:12.716666 16530 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.716833 16505 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.732116 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:12.733569 11367 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:12.735602 16238 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:12.736366 16580 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:12.735857 16631 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:12.737644 16631 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:12.738848 16631 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.695106 16399 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:12.742187 16626 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:17:12.745056 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:12.747385 16623 log.cc:927] Time spent T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:17:12.751842 11543 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:12.795605 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 142586us
I20260709 13:17:12.797794 11514 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:12.799784 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:12.827491 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.738871 16576 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:12.864436 11543 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:12.871496 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:12.702621 16459 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:12.903667 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:12.910650 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:17:12.913554 12085 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.014s	sys 0.003s
W20260709 13:17:12.914335 12085 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.014s	sys 0.004s
W20260709 13:17:12.914767 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 521203us
I20260709 13:17:12.897724 16399 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.913632 16530 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:12.924891 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:12.906275 16576 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:12.913308 16498 consensus_meta.cc:287] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:17:12.947378 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:12.954916 12112 raft_consensus.cc:1240] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:12.957312 12123 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:12.935839 16639 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:12.938145 16530 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.937718 16510 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:12.964267 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:12.946522 16498 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:12.965781 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:12.977036 11397 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:12.978307 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:12.940857 16399 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:12.938894 16453 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:12.976584 16530 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:12.975987 16510 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:12.994400 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 194693us
I20260709 13:17:12.995283 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:12.981448 16631 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:12.976204 16174 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:17:13.025094 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 150404us
I20260709 13:17:13.029515 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:12.993606 16453 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:12.992985 16640 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:12.997942 16510 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:13.079802 16453 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:13.108613 16637 log.cc:927] Time spent T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
W20260709 13:17:13.085868 16641 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:13.091053 16642 consensus_queue.cc:1059] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 13:17:13.101616 16592 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.563s	user 0.004s	sys 0.007s
W20260709 13:17:13.106184 16638 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:17:13.107749 16625 log.cc:927] Time spent T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:17:13.119918 16635 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:17:13.022465 16631 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:13.084432 16421 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:13.136124 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 200613us
I20260709 13:17:13.039135 16557 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:17:13.147727 16646 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:13.152343 16639 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:13.152202 16642 consensus_queue.cc:1243] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:17:13.167654 16399 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:13.148216 16628 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:13.151929 16631 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:13.184428 16554 consensus_meta.cc:287] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 13:17:13.159772 16459 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:13.175451 16646 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:13.229074 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 193797us
W20260709 13:17:13.194921 16336 consensus_meta.cc:287] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.000s
I20260709 13:17:13.201557 16633 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:17:13.238135 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 242791us
I20260709 13:17:13.238533 15705 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:13.242936 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.244989 11548 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:13.250259 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.262049 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:13.262753 11472 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:17:13.231930 16646 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:13.241770 16633 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:13.266564 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.211549 16628 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.229216 16554 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.270668 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.241961 16336 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.277302 11397 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:17:13.291496 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:13.293208 11544 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:13.271446 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 242034us
I20260709 13:17:13.295215 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:13.294047 11365 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:13.297232 11472 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:17:13.298198 16505 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:13.305507 16505 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:13.306603 16505 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.313220 11396 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16. Ops: []
W20260709 13:17:13.316732 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 164864us
W20260709 13:17:13.318696 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 196038us
I20260709 13:17:13.320622 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.321012 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:13.321822 16622 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:13.323535 11514 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.328434 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.322003 16652 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:13.324214 16174 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.326807 16412 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:13.352576 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.322424 16649 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:13.379460 11549 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:13.324085 16459 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:13.330067 16399 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:13.377383 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:13.383239 16653 log.cc:927] Time spent T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.005s
I20260709 13:17:13.362819 16455 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:13.353330 16652 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:13.397997 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.346704 16622 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:13.391126 16648 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:17:13.408672 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.317963 16633 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.411964 11514 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.413326 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:13.415248 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.385381 16577 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:17:13.398890 16657 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.444325 16657 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.416867 16658 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:13.416998 16455 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:13.449163 16510 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.429855 16622 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.430684 16662 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:13.435680 16628 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.451725 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.452028 16510 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.452907 16461 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:13.435784 16554 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:13.436374 16336 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.436904 16652 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.408123 16453 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:13.443586 16577 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:13.454267 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.458312 15080 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:13.376292 16174 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.443162 16501 consensus_queue.cc:1048] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:13.416330 16459 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:13.469540 16657 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.466065 16639 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:13.470242 16652 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:13.471709 16572 consensus_meta.cc:287] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.820s	user 0.006s	sys 0.006s
I20260709 13:17:13.548329 16572 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.550673 16572 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:13.403887 16399 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:13.484076 11542 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:13.485342 16577 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:13.494925 16646 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.557301 16577 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.558573 16646 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:13.498270 15080 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.485723 16530 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.559360 15080 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.560750 11550 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:13.553198 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:13.562263 11396 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:13.520042 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:13.563972 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44028us
I20260709 13:17:13.564730 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.565670 16175 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:13.565804 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:13.566424 16175 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:13.567173 16459 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:13.567627 16459 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.569614 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:13.571117 16175 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:13.558584 16577 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:13.572499 11396 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:17:13.546968 16605 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:13.547441 16554 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:13.478289 16453 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.471271 16631 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.602268 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:13.603806 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.605079 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.605754 11544 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 12.
I20260709 13:17:13.551558 11551 raft_consensus.cc:2368] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10.
I20260709 13:17:13.571544 16175 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:13.529155 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 235172us
I20260709 13:17:13.574021 11514 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:13.580473 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:13.590252 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 290965us
I20260709 13:17:13.560950 16530 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.596324 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:13.612943 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.590548 15080 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.624099 11544 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:13.569640 16459 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:13.629914 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:13.632040 16572 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:13.629913 16646 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:13.639190 16399 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.611366 16175 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:13.655138 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 47181.2us
I20260709 13:17:13.644385 16650 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.647400 16174 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.650067 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:13.651918 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:17:13.652040 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 299546us
I20260709 13:17:13.650496 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.653533 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.601487 16453 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:13.654111 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.660904 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
W20260709 13:17:13.661755 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 49651.1us
I20260709 13:17:13.675310 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.679136 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:13.657474 16572 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:13.678710 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:13.651218 16254 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:17:13.685698 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:13.673313 16442 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.675890 16664 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:13.702456 11475 raft_consensus.cc:1275] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:13.703469 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:13.659020 16399 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.671232 16657 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:17:13.710954 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.706926 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.658900 16650 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:13.707261 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.714124 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.715157 11468 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:13.692482 16563 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:13.690443 16666 consensus_queue.cc:1059] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:17:13.708591 16174 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:13.705104 16632 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:13.708561 16459 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:13.687408 16587 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:17:13.732043 16632 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.763101 16632 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:13.727463 16176 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.732275 16174 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:13.774269 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.798528 16176 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.800328 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.800793 16176 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.801025 11543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 2801) took 1053 ms. Trace:
I20260709 13:17:13.752637 16572 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.801716 16325 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:13.802513 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.803074 11472 raft_consensus.cc:2410] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:17:13.775168 16563 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:13.715744 16579 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:17:13.718494 16664 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:13.721222 16530 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:13.784788 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46176.1us
I20260709 13:17:13.812073 11442 leader_election.cc:400] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10.
I20260709 13:17:13.812387 11442 leader_election.cc:403] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:13.704439 16442 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.793409 16631 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:13.801484 11543 rpcz_store.cc:276] 0709 13:17:12.748001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.890800 (+142799us) service_pool.cc:224] Handling call
0709 13:17:12.891144 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.992742 (+101598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.992800 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:13.024197 (+ 31397us) log.cc:844] Serialized 26 byte log entry
0709 13:17:13.446657 (+422460us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b18004cc3c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da680ee07 00007f8daf669ab9 00007f8da682b69b 00007f8da682993f 00007f8da682bc02 00007f8da94b320b 00007f8da94b8d23 00007f8da94b90d1 00007f8da98d4a0f
0709 13:17:13.660109 (+213452us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:13.660143 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.660202 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.800734 (+140532us) raft_consensus.cc:1685] finished
0709 13:17:13.800761 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.800968 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":185981,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":12611,"spinlock_wait_cycles":71127296,"thread_start_us":314455,"threads_started":2,"wal-append.queue_time_us":653474}
W20260709 13:17:13.805099 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 82508.8us
I20260709 13:17:13.807832 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.839144 11400 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:13.820148 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.822530 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:13.824441 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:13.723708 16622 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.801608 16505 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:13.832432 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.832858 11397 raft_consensus.cc:1240] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:13.833583 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.834371 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:13.835618 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.807981 16174 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:13.838805 11393 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:13.815488 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:13.833482 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:13.845304 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.807495 16325 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:13.849067 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:13.849472 11402 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:13.842907 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:13.846951 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:13.851289 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.853155 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.853627 11514 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:13.859546 16658 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:13.859790 16140 consensus_queue.cc:1048] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:13.860100 16658 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:13.861191 16140 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:13.860908 16658 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.870432 16639 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:13.875573 16421 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:13.850775 16622 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.842208 16577 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:17:13.884017 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.885615 11477 raft_consensus.cc:2410] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:17:13.887348 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:13.841370 16459 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:17:13.904361 16622 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.841766 16663 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:17:13.857338 16640 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:13.926548 12113 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:13.812564 16563 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:13.936774 16664 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.964643 16563 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:13.941039 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:13.965756 11546 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:13.948205 16663 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:13.949657 16666 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:13.950165 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:13.950419 16321 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:13.956287 16442 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:13.866816 16175 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.957847 16632 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.958673 16633 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:13.965767 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.970602 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.969260 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:13.983072 16640 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:13.984520 16582 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:13.975281 16662 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:13.993177 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.999471 16663 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.003321 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:13.984169 16453 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:13.973321 16577 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 9. Current term is 10.
I20260709 13:17:14.011868 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.028789 11544 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:14.010089 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.010094 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:14.008040 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.030699 11546 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:14.029779 11549 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:14.000603 16438 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:14.000782 16657 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.040069 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.040660 11396 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:14.041461 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.042042 11397 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:17:14.043853 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.006958 16582 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.044967 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:14.025960 16526 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.044624 16657 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.047780 11396 raft_consensus.cc:2393] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 18: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:14.048983 16605 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.052407 16660 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.989037 16666 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.063665 11395 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:14.029136 16672 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.031102 16399 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:14.069612 16672 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.074218 16672 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.074779 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:14.080125 16632 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.083379 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:14.084236 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:14.084259 16439 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.085253 16439 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.085605 16663 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.085760 16321 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.086194 16321 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.090888 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.028182 16571 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:17:14.101727 16576 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:14.102185 16175 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:14.064858 16658 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:14.015017 16640 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.093500 16321 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.085618 16439 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.096421 16582 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.099264 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.027864 16662 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.104070 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.108788 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.068104 16605 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.065061 16526 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.147603 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.102943 16658 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.102198 16576 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:14.157460 11475 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:17:14.146672 16526 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.163409 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.136389 16605 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.157634 16658 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:14.179615 16576 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.180203 16640 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.180661 16321 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.182396 16662 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.183545 16453 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.089468 16660 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.202564 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.203230 12123 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:14.204649 11441 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:14.205174 16321 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.211239 16453 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:14.211789 16453 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.212483 16453 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:14.192662 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.196028 16572 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:14.218376 16572 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.195807 16579 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:14.214401 11472 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:17:14.229092 11472 pending_rounds.cc:85] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 8
I20260709 13:17:14.229406 11472 pending_rounds.cc:107] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:17:14.229704 11472 raft_consensus.cc:2889] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:14.192179 11468 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
W20260709 13:17:14.201893 16652 consensus_meta.cc:287] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.004s
I20260709 13:17:14.213840 16666 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.242965 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.243468 11474 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:17:14.243860 11474 pending_rounds.cc:85] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:17:14.244098 11474 pending_rounds.cc:107] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:17:14.244451 11474 raft_consensus.cc:2889] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:14.232160 11468 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:14.220744 16576 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:14.241334 16665 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:14.254010 16562 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:17:14.244001 16579 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.262493 16665 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.292315 16439 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.294000 16572 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.292946 16657 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.295895 16439 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.321193 16579 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.322384 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.324198 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:14.325986 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:14.334800 11471 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:14.302121 16660 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.330374 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.339068 11470 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.282977 16174 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.348415 16665 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.351377 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.348795 16453 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.365909 16453 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.359578 11477 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:14.369210 16510 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:14.369604 16510 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:14.371284 11469 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:17:14.373140 16582 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:14.373291 11469 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:17:14.379052 16450 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:14.380620 16174 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.388603 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:14.391983 16650 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.395458 16563 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:14.395916 16563 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:14.396317 16563 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:17:14.396744 16694 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:17:14.410918 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.420692 11548 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:14.421229 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.422049 11545 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:14.436817 16176 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.437417 16176 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.370409 16510 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.446183 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.449353 11548 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.450567 16510 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:14.450630 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.451536 11395 raft_consensus.cc:2393] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:14.456408 11545 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:14.460691 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.461663 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.462647 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:14.465052 11470 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:14.466406 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:14.468178 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 57551.1us
I20260709 13:17:14.472443 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.473031 11393 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:14.473919 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.474939 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.479043 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.479368 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:14.479727 11394 raft_consensus.cc:2368] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16.
I20260709 13:17:14.480026 11395 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:17:14.480368 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.480939 11399 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:14.483842 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.485358 16605 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.447834 16336 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:14.487661 16605 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.407279 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:14.493834 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.494944 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.495322 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.503957 11542 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:14.496218 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.501505 11393 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:14.497268 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.497900 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.501466 11399 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:17:14.513595 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.514094 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.514842 11547 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.517351 11546 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:14.517181 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.474849 16175 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:14.522563 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.516914 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.523890 11399 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:14.459772 16577 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:17:14.531222 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:14.532567 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 125388us
I20260709 13:17:14.529031 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.529474 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.541576 16650 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:14.530452 16336 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.534634 16689 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:14.542928 16633 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.542809 16336 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.543390 16633 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.543785 16633 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.544831 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:14.545240 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.471154 16666 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:17:14.548918 16572 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:14.549314 16450 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:14.552320 11365 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:14.553071 16459 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:14.553818 16639 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:14.530638 16640 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:14.559335 16176 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.567180 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.592859 11468 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:14.593094 16526 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.599514 16576 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:14.600057 16576 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:14.600881 16576 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.602036 16576 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.529090 11442 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:17:14.576581 11367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.584316 16140 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:14.585902 16421 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:17:14.588515 16450 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:14.589179 16175 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:14.613291 12112 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:17:14.628275 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 99241.3us
I20260709 13:17:14.629730 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.631454 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.536145 16658 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.639598 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.630940 16662 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:14.632145 16510 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:14.639108 16175 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:14.643410 16175 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:14.644713 16658 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:14.639293 16174 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:14.644320 16175 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.649356 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.650034 11398 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:14.651373 11544 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:14.653595 11542 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:14.656986 11548 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:14.630388 16140 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.658215 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:14.636219 16526 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.636891 16639 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:14.664654 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.657683 16174 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:14.666944 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.665548 11398 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:14.670339 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:14.672758 16640 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:14.672997 16642 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.673333 16640 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:14.673595 16642 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.674147 11468 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:14.675356 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.677513 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.678303 11398 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:14.664110 16665 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:14.685046 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:14.665136 16639 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:14.685340 11367 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.687443 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.688381 16450 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:14.691329 11442 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.674337 16640 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.690397 16658 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:14.695051 16665 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:14.697052 16655 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:14.697563 16655 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.698340 16655 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.701752 16642 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.704388 11367 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.704936 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.705749 16665 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:14.707217 11442 leader_election.cc:400] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16.
I20260709 13:17:14.716993 11442 leader_election.cc:403] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:17:14.707751 16580 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:14.718451 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:14.719383 11442 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:14.689139 16140 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.721899 16665 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.724433 16639 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.727180 16662 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:14.727873 16662 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:14.728441 16639 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:14.728969 16662 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.743235 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:14.750439 11547 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:14.764204 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:14.767102 11547 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:14.771446 11468 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:17:14.784279 16450 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:14.785730 16666 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:17:14.749841 16412 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:17:14.764292 16633 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.749513 16577 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:17:14.788312 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.804136 11394 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:14.808450 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.810009 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:14.811746 11547 raft_consensus.cc:1240] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
W20260709 13:17:14.787195 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 280431us
I20260709 13:17:14.812668 16336 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:14.813270 16336 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.815209 16277 consensus_queue.cc:1059] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:14.816350 16277 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:14.816787 16277 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:14.817632 16277 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.820878 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.822569 11550 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 16: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:14.823298 16579 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.823830 16579 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.825853 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.826424 12112 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:14.826614 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:14.814167 16336 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.827220 11471 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:14.828675 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:14.830227 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:14.832089 16277 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:14.834372 16254 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:14.834601 16579 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.836201 16423 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:14.836753 16423 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.837142 16423 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:14.834959 16254 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.845880 16254 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:14.851936 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.852869 11547 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:14.706461 16563 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.860893 16423 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.862111 16563 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:14.866427 16453 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.866971 16453 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:14.869117 16453 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:14.777058 16658 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:14.870038 16658 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.871817 16254 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.872298 16658 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.872959 16577 raft_consensus.cc:697] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:14.873899 16577 consensus_queue.cc:237] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:14.878111 11548 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
W20260709 13:17:14.879568 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 73819.4us
I20260709 13:17:14.881031 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:17:14.881809 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 51852.3us
I20260709 13:17:14.881901 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:14.882648 12123 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:17:14.860167 16665 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16.
I20260709 13:17:14.878837 12113 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:14.888489 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:14.780408 16689 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:14.894130 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.874233 16254 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:14.904055 12113 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:17:14.906829 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:14.908865 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:14.909298 11551 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:14.749734 16174 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 13:17:14.907509 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:14.860239 16633 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:14.949268 16707 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:17:14.940042 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:14.986868 11296 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 230) took 58998 ms (client timeout 15000 ms). Trace:
I20260709 13:17:14.992447 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:14.997030 16649 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:15.007992 16423 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.009210 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:15.009887 11550 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.020467 11550 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
W20260709 13:17:15.020747 16704 log.cc:927] Time spent T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 13:17:15.024269 16699 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
W20260709 13:17:14.992048 11296 rpcz_store.cc:269] 0709 13:16:15.988173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.988466 (+   293us) service_pool.cc:224] Handling call
0709 13:17:14.986789 (+58998323us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:10.363972 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:10.432101 (+ 68129us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:10.432166 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:17:10.432200 (+    34us) tablet.cc:662] Decoding operations
0709 13:17:10.432605 (+   405us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:10.432694 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:10.432722 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:10.433019 (+   297us) tablet.cc:701] Row locks acquired
0709 13:17:10.433069 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:17:10.433259 (+   190us) write_op.cc:270] Start()
0709 13:17:10.433405 (+   146us) write_op.cc:276] Timestamp: P: 1783603030433188 usec, L: 0
0709 13:17:10.433429 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:10.434153 (+   724us) log.cc:844] Serialized 532 byte log entry
0709 13:17:11.183063 (+748910us) op_driver.cc:464] REPLICATION: finished
0709 13:17:11.423777 (+240714us) write_op.cc:301] APPLY: starting
0709 13:17:11.432840 (+  9063us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:11.434009 (+  1169us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:11.434141 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:11.434915 (+   774us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:11.435161 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:17:11.435242 (+    81us) write_op.cc:312] APPLY: finished
0709 13:17:11.435467 (+   225us) log.cc:844] Serialized 25 byte log entry
0709 13:17:11.435665 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.435767 (+   102us) write_op.cc:454] Released schema lock
0709 13:17:11.435819 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:11.491252 (+ 55433us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.494602 (+  3350us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2067570,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":169595,"apply.run_cpu_time_us":3730,"apply.run_wall_time_us":44696,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":449555,"prepare.run_cpu_time_us":51800,"prepare.run_wall_time_us":146489,"raft.queue_time_us":350657,"raft.run_cpu_time_us":33313,"raft.run_wall_time_us":222777,"replication_time_us":722111,"thread_start_us":529254,"threads_started":4,"wal-append.queue_time_us":117766,"wal-append.run_cpu_time_us":84492,"wal-append.run_wall_time_us":1901256}]]}
I20260709 13:17:15.014374 11300 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.029392 11300 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 14, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.030496 11300 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.032545 16576 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:15.032976 16576 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:15.033759 16576 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.034627 16576 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:15.040620 11300 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 17 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.042605 11300 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.065407 11542 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
W20260709 13:17:15.071524 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.080014 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:15.080646 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:15.081207 11473 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.069758 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:15.083094 11442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.083756 11300 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.085270 11300 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.086030 11394 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:15.086727 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:15.087158 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.085291 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.087972 11396 raft_consensus.cc:2368] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18.
I20260709 13:17:15.088788 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:15.089527 11398 raft_consensus.cc:2368] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19.
I20260709 13:17:15.091490 11300 catalog_manager.cc:5697] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 9, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 9 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.096359 11300 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.089602 16412 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:15.107878 11300 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.114077 11473 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:15.120078 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.124727 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.127446 11300 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 11, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.128492 12085 leader_election.cc:400] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19.
I20260709 13:17:15.159035 12085 leader_election.cc:403] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:17:15.166592 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.169114 11300 catalog_manager.cc:5697] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 10, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.179459 11365 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:15.180202 11300 catalog_manager.cc:5697] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 11, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 11 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.159422 16487 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:15.195779 11300 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.178324 16715 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:17:15.174817 16405 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19.
W20260709 13:17:15.205665 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 82351us
I20260709 13:17:15.207603 11516 leader_election.cc:400] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18.
I20260709 13:17:15.209226 11516 leader_election.cc:403] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:15.204650 11300 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 15, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.233122 16450 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:15.236039 11300 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.203799 16689 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:15.209002 15705 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:15.239781 16175 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:15.200860 16702 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:17:15.247614 16718 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:17:15.253577 16717 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:17:15.254819 16719 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:17:15.257647 16650 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:17:15.310079 11300 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:15.318481 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 46188.4us
W20260709 13:17:15.318668 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43874.7us
I20260709 13:17:15.324363 11551 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:15.325819 11300 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.326936 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44877us
I20260709 13:17:15.331630 11300 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.341840 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.344039 16722 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:15.344515 16722 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:15.344702 16321 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:15.345409 16722 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.346678 16722 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.353315 11472 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:15.361034 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.363345 16566 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:15.369177 11542 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:15.324227 15705 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:15.298003 16689 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:15.329330 16666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.011s	sys 0.025s
I20260709 13:17:15.378157 16633 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:15.380438 12113 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:17:15.380530 11300 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 13, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:15.386029 15705 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.394367 11300 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 14 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:15.385197 16666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.011s	sys 0.025s
W20260709 13:17:15.395642 16714 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.547s	user 0.000s	sys 0.005s
I20260709 13:17:15.394346 16712 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:17:15.413975 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.414613 11395 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:15.418352 11542 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:17:15.421044 11395 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:15.419584 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:15.424301 11300 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.387642 16665 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:17:15.435705 11300 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 13, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.438879 16713 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:15.446058 16713 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:15.446440 16713 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:15.428026 16658 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:17:15.411254 16566 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:17:15.471464 16566 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:15.471901 16566 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:15.472805 16566 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:15.473978 16689 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.487896 11394 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:15.491051 12113 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:15.493151 11395 raft_consensus.cc:1275] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:17:15.496083 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54807.8us
I20260709 13:17:15.499159 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:15.499948 11543 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:15.509002 11475 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:15.512508 11300 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.515498 11300 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 12, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:15.519605 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 68830.6us
I20260709 13:17:15.531486 11300 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.543526 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:15.536614 16711 consensus_queue.cc:1048] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:17:15.547067 16321 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:15.548214 16602 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:15.564390 11543 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:15.569936 11544 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:15.575079 11300 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 5 to 6, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 6 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.578568 11473 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:15.579129 11473 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:15.580075 11473 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.534945 16662 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:17:15.598565 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 71985.7us
I20260709 13:17:15.555886 16277 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:15.606642 11300 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 10, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.612514 16277 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.614596 16277 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.614758 16650 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18.
I20260709 13:17:15.609092 11300 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 4 to 7, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 7 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.618741 16577 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:15.652354 11473 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:15.652457 16726 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:15.653867 16632 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:15.661607 11394 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:15.661911 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:15.662569 11542 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:15.670841 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:15.645061 16566 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.681622 16713 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.684190 11300 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 14, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.685847 11300 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.688834 16713 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.689795 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:15.690598 11544 raft_consensus.cc:2393] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 11: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:15.705616 11476 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:17:15.706316 11476 pending_rounds.cc:85] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 8
I20260709 13:17:15.706548 11476 pending_rounds.cc:107] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:17:15.706941 11476 raft_consensus.cc:2889] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:15.708352 11473 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:17:15.708756 11473 pending_rounds.cc:85] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Aborting all ops after (but not including) 6
I20260709 13:17:15.708987 11473 pending_rounds.cc:107] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:17:15.709210 11473 raft_consensus.cc:2889] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:15.710268 16666 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.710515 16652 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:17:15.710785 16666 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.712821 16666 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:15.713246 16740 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:17:15.715559 16722 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:15.714572 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:15.717662 11544 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:15.719205 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:17:15.733094 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 89656.9us
I20260709 13:17:15.736438 11300 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.756304 11300 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.732651 16577 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:15.759116 16737 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.137s	user 0.000s	sys 0.005s
I20260709 13:17:15.792932 16175 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.793514 16175 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.767323 11300 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 11, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.795835 16175 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:15.796244 11300 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.773805 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 103052us
I20260709 13:17:15.798614 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:15.799484 11473 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
W20260709 13:17:15.774463 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 93951.1us
I20260709 13:17:15.776316 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.801929 11396 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:15.802481 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:15.803582 16175 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:15.804100 16175 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.804515 16175 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.806291 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.808043 11300 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 11, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.816388 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:15.770378 16577 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:15.823238 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 87000.1us
I20260709 13:17:15.817682 15705 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:15.831501 11398 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:17:15.838019 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.833617 15705 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:15.850311 15705 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:15.855516 16726 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.861886 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:15.863051 11473 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:15.869787 15705 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.874684 15705 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.879388 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:15.880210 11475 raft_consensus.cc:2393] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:17:15.832370 16577 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:15.865613 16735 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:15.831939 16566 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:17:15.881300 16672 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:15.888023 16739 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:15.856122 16726 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:15.927829 16741 log.cc:927] Time spent T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:17:15.934600 16726 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.916178 16672 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.947362 16672 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:15.911415 16735 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:15.948724 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:15.953394 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:15.955027 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:15.973341 11550 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
W20260709 13:17:15.977942 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65908.3us
I20260709 13:17:15.987015 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:15.987741 11395 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:15.991451 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:15.993556 16672 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:15.993978 16672 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.994320 16672 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.003494 16450 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:16.004102 16450 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:15.992930 11367 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:16.007658 16657 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:16.008237 16657 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:15.949272 16735 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.015306 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:17:16.020228 16743 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:17:16.011281 16175 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.029740 16175 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:16.009089 16657 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.031080 16450 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:16.059669 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:16.060250 11475 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:16.100464 16746 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 13:17:16.105857 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 126831us
I20260709 13:17:16.058604 16577 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.111232 16577 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.113425 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:16.118854 11475 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:16.122781 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:16.123370 11544 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:16.123623 11514 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:16.139360 11476 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:16.140167 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:16.142072 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:16.142652 11399 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:17:16.144434 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:16.145205 11398 raft_consensus.cc:2393] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:16.074414 16672 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.161619 11544 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:16.162873 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:16.135183 16605 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:16.208643 11470 raft_consensus.cc:2368] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17.
W20260709 13:17:16.190829 16743 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:16.190699 16577 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:16.223788 11399 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:16.211024 16605 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:16.224486 16453 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:17:16.255223 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55204.3us
I20260709 13:17:16.226676 16577 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:16.294961 16577 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.298437 16605 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.299628 16421 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.300133 16421 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.302098 16421 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.303731 16672 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.306720 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:16.262024 16453 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.316198 11547 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:16.317266 11545 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:16.335060 16453 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.337129 16649 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:16.337806 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:16.338461 11477 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:16.343465 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:16.344323 11551 raft_consensus.cc:2393] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 14: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:16.345345 11514 leader_election.cc:400] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17.
I20260709 13:17:16.345708 11514 leader_election.cc:403] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:16.346266 16723 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.351529 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:16.356596 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:16.357842 16659 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:16.358279 16659 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.358615 16659 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:16.369765 16723 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17.
I20260709 13:17:16.345474 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:16.379374 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:16.380254 11393 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.374670 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:16.384815 16631 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:16.403680 11547 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:16.406874 16453 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:16.407557 16453 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.415900 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:16.416846 11399 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:16.418426 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:16.419126 11545 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:16.420284 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:16.421020 11393 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:16.421137 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:16.417032 16453 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.423173 16724 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:16.424337 16713 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:16.424616 16666 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:16.452567 16666 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:16.479481 16666 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:16.455874 16713 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:16.483691 11551 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:16.495806 16659 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.497762 16659 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.491117 16713 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.517488 16666 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.520520 16666 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:16.573638 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 88348.2us
I20260709 13:17:16.579018 11394 raft_consensus.cc:2368] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 14. Current term is 15.
I20260709 13:17:16.584112 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:16.585608 11470 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:16.597579 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:16.599862 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:16.598802 11544 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:16.626649 16642 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.627542 16642 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:16.630249 16650 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.631117 11442 leader_election.cc:400] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 14. Current term is 15.
I20260709 13:17:16.631773 11442 leader_election.cc:403] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:16.633376 16642 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:16.639216 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 61835.1us
I20260709 13:17:16.642949 11393 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:16.646154 16439 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:16.631198 16650 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.653419 11547 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:16.659247 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:16.659994 11395 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:16.666836 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:16.668278 11394 raft_consensus.cc:2393] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:16.669932 11516 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:16.681102 16723 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:16.681185 16650 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:16.681663 16723 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.682158 16723 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:16.684113 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:16.685119 16672 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:16.687656 16650 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.689041 16666 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:16.689936 16650 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:16.691556 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:16.692453 12113 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:16.694626 11469 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:16.697146 16650 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:16.698678 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:16.699594 12113 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:17:16.701045 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:16.704416 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:16.705394 16642 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:16.705834 16642 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.706297 16642 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.721138 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:16.714923 16650 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:16.724365 11549 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:16.728853 16650 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.737649 16650 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.740782 11551 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:16.758951 11544 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:16.770763 11476 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:16.779462 11470 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:16.782604 16175 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:16.782897 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:16.783135 16453 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 14. Current term is 15.
I20260709 13:17:16.790843 11395 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:16.792387 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:16.796756 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:17:16.797854 16582 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:17:16.774909 16723 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.824576 16666 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:16.825277 16666 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:16.836436 11545 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:17:16.826623 16666 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.865803 16642 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:16.866271 16497 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:16.868403 11393 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:16.866838 16497 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.870591 16497 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:16.870692 11396 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:16.876667 11365 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:16.879774 16336 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:16.883738 16723 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:16.885483 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:16.887108 16336 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:16.888290 16336 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.889480 11396 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:17:16.900784 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 118417us
I20260709 13:17:16.906179 16642 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:16.906762 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:16.907441 12113 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.913081 16605 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:16.923750 16582 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.927101 11549 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:16.927568 16582 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.929703 16582 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:16.936416 16769 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:17:16.941447 12113 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:16.947257 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:16.945132 16453 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:17:16.950796 11366 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:16.968379 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:16.952657 16672 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:17:16.971627 11514 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:16.971874 16672 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:16.972481 16672 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:16.973655 16672 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:16.974881 16175 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:17.000558 15705 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:17.002707 15705 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:16.996310 16672 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:16.984119 16453 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:17:17.014374 11399 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:17.015736 15705 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.012171 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:17.027266 16453 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:17:17.054538 11286 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 147201d22a694ca59f6e014f083a0892 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.073508 16650 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.076355 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:17.076603 16650 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:17.066736 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 54513.2us
I20260709 13:17:17.077773 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:17.092509 12112 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:17:17.094966 16773 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:17:17.094811 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:17.093191 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:17.095767 11548 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 17: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:17:17.101137 11547 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:17.103250 16662 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:17.106549 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:17.107512 11542 raft_consensus.cc:2410] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:17:17.111107 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:17.111989 16723 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:17.185890 16643 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.186563 16643 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.193145 16643 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:17.197607 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:17.198165 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:17.199100 11547 raft_consensus.cc:2410] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:17:17.184273 15705 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.206207 16771 log.cc:927] Time spent T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:17:17.210471 15705 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.221271 11396 raft_consensus.cc:2393] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 18: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:17:17.235457 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47406us
I20260709 13:17:17.269315 15705 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:17.273321 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:17.273945 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:17.273979 12122 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:17.279531 16702 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:17.293078 16702 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:17.321478 11475 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:17.323412 15705 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:17.327677 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:17.328420 11549 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
W20260709 13:17:17.329933 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56180.5us
I20260709 13:17:17.329384 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:17.335453 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:17.340479 15705 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:17.340888 15705 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:17.341163 15705 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:17.343181 16421 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:17.346086 11367 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:17.346904 16336 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:17.367254 11544 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:17.372545 16662 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:17.376323 15705 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.379366 15705 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:17.385214 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:17.385841 12122 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:17.394971 12122 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:17:17.397370 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:17.398123 16439 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:17.398617 16439 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:17.399682 16439 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.412621 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:17.413298 11549 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:17:17.442205 11300 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 246) took 16262 ms (client timeout 15000 ms). Trace:
I20260709 13:17:17.436518 11549 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:17:17.446713 11396 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:17.448206 16662 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:17.451848 11295 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 7 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 10 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.455080 12122 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:17.465543 11544 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:17:17.442678 11300 rpcz_store.cc:269] 0709 13:17:01.179753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.184570 (+  4817us) service_pool.cc:224] Handling call
0709 13:17:17.441997 (+16257427us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:16.286626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:16.304616 (+ 17990us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:16.304652 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:17:16.304673 (+    21us) tablet.cc:662] Decoding operations
0709 13:17:16.326197 (+ 21524us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:16.326279 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:16.326304 (+    25us) tablet.cc:685] Acquiring locks for 36 operations
0709 13:17:16.332449 (+  6145us) tablet.cc:701] Row locks acquired
0709 13:17:16.332465 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:17:16.332604 (+   139us) write_op.cc:270] Start()
0709 13:17:16.332717 (+   113us) write_op.cc:276] Timestamp: P: 1783603036332560 usec, L: 0
0709 13:17:16.332735 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:17:16.333396 (+   661us) log.cc:844] Serialized 13904 byte log entry
0709 13:17:16.513173 (+179777us) op_driver.cc:464] REPLICATION: finished
0709 13:17:16.525240 (+ 12067us) write_op.cc:301] APPLY: starting
0709 13:17:16.525338 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:16.638649 (+113311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:16.638879 (+   230us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:17.098786 (+459907us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:17.153138 (+ 54352us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=72,key_file_lookups=72,delta_file_lookups=72,mrs_lookups=0
0709 13:17:17.157837 (+  4699us) write_op.cc:312] APPLY: finished
0709 13:17:17.183439 (+ 25602us) log.cc:844] Serialized 308 byte log entry
0709 13:17:17.210643 (+ 27204us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:17.242483 (+ 31840us) write_op.cc:454] Released schema lock
0709 13:17:17.242596 (+   113us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:17.247815 (+  5219us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:17.247857 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":602466,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":11772,"apply.run_cpu_time_us":56913,"apply.run_wall_time_us":722527,"cfile_cache_hit":146,"cfile_cache_hit_bytes":154679,"num_ops":36,"prepare.queue_time_us":291993,"prepare.run_cpu_time_us":36971,"prepare.run_wall_time_us":154339,"raft.queue_time_us":15076,"raft.run_cpu_time_us":3931,"raft.run_wall_time_us":14595,"replication_time_us":180203,"spinlock_wait_cycles":2304,"thread_start_us":366824,"threads_started":4,"wal-append.queue_time_us":138143}]]}
I20260709 13:17:17.482302 16336 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:17:17.500965 11542 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:17.508391 15705 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:17.522048 16579 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:17.599095 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:17.599854 11398 raft_consensus.cc:606] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:17.602777 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:17.603369 11398 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:17.603682 11398 raft_consensus.cc:614] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:17.607100 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:17.607827 11398 raft_consensus.cc:606] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:17.616765 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:17.617429 11398 raft_consensus.cc:606] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:17.620910 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:17.621642 11398 raft_consensus.cc:606] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:17.622028 11398 raft_consensus.cc:614] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:17.623108 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:17.624943 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:17.726629 11292 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 13, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.728612 11292 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 17 to 19, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 19 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.731112 11292 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.740916 11292 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.767179 11292 catalog_manager.cc:5697] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.772987 11292 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.776525 11292 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 15. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.778378 11292 catalog_manager.cc:5697] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 11, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 11 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.780465 11292 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 12, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 12 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.782378 11292 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.811486 11292 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.815382 11292 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.818123 11292 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:17.826124 11295 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 217) took 37519 ms (client timeout 15000 ms). Trace:
I20260709 13:17:17.825120 11292 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.828038 11292 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:17.826570 11295 rpcz_store.cc:269] 0709 13:16:40.306375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.309299 (+  2924us) service_pool.cc:224] Handling call
0709 13:16:40.697867 (+388568us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:17.826029 (+37128162us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:17.648539 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:17.649181 (+   642us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:17.649229 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:17:17.649247 (+    18us) tablet.cc:662] Decoding operations
0709 13:17:17.649619 (+   372us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:17.649716 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:17.649746 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:17.650022 (+   276us) tablet.cc:701] Row locks acquired
0709 13:17:17.650066 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:17:17.650227 (+   161us) write_op.cc:270] Start()
0709 13:17:17.650375 (+   148us) write_op.cc:276] Timestamp: P: 1783603037650169 usec, L: 0
0709 13:17:17.650396 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:17.651112 (+   716us) log.cc:844] Serialized 532 byte log entry
0709 13:17:17.663943 (+ 12831us) op_driver.cc:464] REPLICATION: finished
0709 13:17:17.664544 (+   601us) write_op.cc:301] APPLY: starting
0709 13:17:17.664648 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:17.665417 (+   769us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:17.665509 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:17.666178 (+   669us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:17.666351 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:17:17.666401 (+    50us) write_op.cc:312] APPLY: finished
0709 13:17:17.666593 (+   192us) log.cc:844] Serialized 25 byte log entry
0709 13:17:17.666767 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:17.666844 (+    77us) write_op.cc:454] Released schema lock
0709 13:17:17.666883 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:17.667840 (+   957us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:17.667873 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1570058,"reported_tablets":51,"spinlock_wait_cycles":35578112,"child_traces":[["op",{"apply.queue_time_us":367,"apply.run_cpu_time_us":2691,"apply.run_wall_time_us":3253,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":31000,"prepare.run_cpu_time_us":2901,"prepare.run_wall_time_us":6194,"replication_time_us":13377,"thread_start_us":30380,"threads_started":1}]]}
I20260709 13:17:17.836863 11292 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 10 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.844707 11292 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 16, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 16 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.846731 11292 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.848449 11292 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 18, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.850064 11292 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.856513 11292 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.863969 11292 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.865739 11292 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.868021 11292 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 8, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 8 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.890967 11292 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.903234 11292 catalog_manager.cc:5697] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 19, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 19 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.923731 16336 raft_consensus.cc:993] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:17.924300 16336 raft_consensus.cc:1081] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:17.926340 11471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:17.926954 11471 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.927299 11471 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:17.931083 11292 catalog_manager.cc:5697] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 15 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.933109 11292 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.934911 11292 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:17.936539 11471 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.936880 11292 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:17.938395 11292 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.939764 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:17.940387 11398 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:17.940670 11398 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:17.940810 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:17.941432 11547 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:17.942018 11471 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:17.941506 11398 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:17.955423 12122 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:17.943738 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:17.956604 12112 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:17.949245 11547 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:17.964402 11548 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:17.965369 16582 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:17.966060 16582 consensus_queue.cc:1243] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:17:17.967161 16421 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:17.971915 12122 pending_rounds.cc:85] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 12
I20260709 13:17:17.972380 12122 pending_rounds.cc:107] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:17:17.972681 12122 raft_consensus.cc:2889] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:17.975024 11398 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:17.981249 16579 raft_consensus.cc:993] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:17.981922 16579 raft_consensus.cc:1081] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:17.987787 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:17.988552 11475 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.988934 11475 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:17.988992 12112 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:17.996043 11398 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:17.997951 11441 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:18.000813 16666 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:18.001364 16666 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:18.002293 16666 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:18.010250 11475 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.014783 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:18.015364 11396 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:18.015600 11396 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:18.022840 11475 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:18.023662 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:18.024477 11544 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.016238 11396 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.026261 11396 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.040062 16557 raft_consensus.cc:993] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:18.040849 16579 raft_consensus.cc:1081] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:18.085503 11396 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:18.086750 11442 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:18.086812 11544 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:18.089659 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:18.090252 11475 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.090569 11475 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.091346 16666 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:18.091898 16666 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:18.092859 16666 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.147622 11475 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:18.149936 11475 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:18.150897 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:18.151499 12122 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.155341 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:18.155941 11396 raft_consensus.cc:3055] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:18.156212 11396 raft_consensus.cc:740] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:18.156940 11396 consensus_queue.cc:260] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:18.157990 11396 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.172993 12122 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:18.174124 11441 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:18.175215 16666 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:18.175827 16666 raft_consensus.cc:697] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:18.176267 11396 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:18.177029 16666 consensus_queue.cc:237] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:18.327726 16577 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.328301 16577 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.330376 16577 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:18.333983 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:18.336668 11292 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 247) took 1426 ms. Trace:
I20260709 13:17:18.337168 11292 rpcz_store.cc:276] 0709 13:17:16.910422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.910754 (+   332us) service_pool.cc:224] Handling call
0709 13:17:18.336578 (+1425824us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:18.081142 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:18.081737 (+   595us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:18.081775 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:17:18.081794 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:18.088331 (+  6537us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:18.088398 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:18.088431 (+    33us) tablet.cc:685] Acquiring locks for 31 operations
0709 13:17:18.093594 (+  5163us) tablet.cc:701] Row locks acquired
0709 13:17:18.093610 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:17:18.093754 (+   144us) write_op.cc:270] Start()
0709 13:17:18.093863 (+   109us) write_op.cc:276] Timestamp: P: 1783603038093710 usec, L: 0
0709 13:17:18.093878 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:17:18.124137 (+ 30259us) log.cc:844] Serialized 11849 byte log entry
0709 13:17:18.128000 (+  3863us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.128622 (+   622us) write_op.cc:301] APPLY: starting
0709 13:17:18.128731 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.149399 (+ 20668us) spinlock_profiling.cc:243] Waited 944 us on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9914e28 00007f8dad34f581 00007f8dad34ed58 00007f8dad3539f1 00007f8dad3539a9 00007f8dad353939 00007f8dad353901 00007f8dad352b2d 000000000038f354 000000000038f199
0709 13:17:18.188841 (+ 39442us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b18003ffc40. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da682c522 00007f8da682e7c1 00007f8da682e779 00007f8da682e709 00007f8da682e6d1 00007f8da682d9cd 000000000038f354 000000000038f199 00007f8da680f83a
0709 13:17:18.192440 (+  3599us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.192471 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:18.250442 (+ 57971us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:18.250784 (+   342us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 13:17:18.250831 (+    47us) write_op.cc:312] APPLY: finished
0709 13:17:18.251396 (+   565us) log.cc:844] Serialized 268 byte log entry
0709 13:17:18.269696 (+ 18300us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.269930 (+   234us) write_op.cc:454] Released schema lock
0709 13:17:18.269999 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:18.274540 (+  4541us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.274577 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":50790,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":348,"apply.run_cpu_time_us":36974,"apply.run_wall_time_us":145855,"cfile_cache_hit":124,"cfile_cache_hit_bytes":129642,"num_ops":31,"prepare.queue_time_us":1206,"prepare.run_cpu_time_us":9614,"prepare.run_wall_time_us":68424,"replication_time_us":33923,"spinlock_wait_cycles":89005184}]]}
I20260709 13:17:18.342036 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:18.342760 11548 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:18.344178 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:18.344871 16577 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:18.345211 16577 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.345533 16577 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:18.352504 11294 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 4 to 8, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 8 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.359514 11294 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 8 to 9, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.371842 16577 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.384814 16577 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:18.385210 11294 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 6 to 10, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 10 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:18.389736 11294 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.390669 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:18.399405 11549 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:18.391850 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:18.407460 11549 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:18.409868 11294 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 8 to 10, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 10 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.426275 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:18.427665 16577 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:18.428170 16577 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:18.429000 16577 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.471931 11295 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 14 to 19, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 19 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.474196 11295 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:18.475839 11295 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 8 to 11, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 11 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:18.478137 11295 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 16, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:18.479820 11295 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 12, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:18.487599 16336 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:18.488169 16336 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:18.488444 11295 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:18.490146 16336 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:18.492623 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:18.490437 11295 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 14 to 17, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.494521 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:18.495386 11547 raft_consensus.cc:2368] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:17:18.498041 11367 leader_election.cc:400] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:17:18.498342 11367 leader_election.cc:403] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:18.499094 16579 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:18.503763 11396 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:18.507424 16666 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:18.508618 12122 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:18.511971 16577 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:18.525928 11398 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:18.527768 16666 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:17:18.541958 11294 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 231) took 29333 ms (client timeout 15000 ms). Trace:
I20260709 13:17:18.550617 16579 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 12. Current term is 13.
I20260709 13:17:18.574591 11548 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:18.585595 16577 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
W20260709 13:17:18.552390 11294 rpcz_store.cc:269] 0709 13:16:49.211422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.211717 (+   295us) service_pool.cc:224] Handling call
0709 13:17:18.541860 (+29330143us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:18.427638 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:18.428158 (+   520us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:18.428190 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:17:18.428203 (+    13us) tablet.cc:662] Decoding operations
0709 13:17:18.428845 (+   642us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:18.428892 (+    47us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:18.428924 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:17:18.429423 (+   499us) tablet.cc:701] Row locks acquired
0709 13:17:18.429437 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:17:18.429574 (+   137us) write_op.cc:270] Start()
0709 13:17:18.429686 (+   112us) write_op.cc:276] Timestamp: P: 1783603038429530 usec, L: 0
0709 13:17:18.429715 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:17:18.430301 (+   586us) log.cc:844] Serialized 2039 byte log entry
0709 13:17:18.435748 (+  5447us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.436201 (+   453us) write_op.cc:301] APPLY: starting
0709 13:17:18.436276 (+    75us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.438666 (+  2390us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.438761 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:18.441445 (+  2684us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:18.441617 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:17:18.441664 (+    47us) write_op.cc:312] APPLY: finished
0709 13:17:18.441905 (+   241us) log.cc:844] Serialized 57 byte log entry
0709 13:17:18.442067 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.442159 (+    92us) write_op.cc:454] Released schema lock
0709 13:17:18.442199 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:18.442561 (+   362us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.442590 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3934259,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":225,"apply.run_cpu_time_us":6306,"apply.run_wall_time_us":6306,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":3015,"prepare.run_wall_time_us":7566,"replication_time_us":5875,"spinlock_wait_cycles":35200}]]}
I20260709 13:17:18.623102 11298 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:18.625010 11298 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:18.628016 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:18.648121 12122 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:18.649821 16577 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:18.688176 11298 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 10 to 11, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 11 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:18.690758 11298 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:18.705667 11298 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:18.776366 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 91187us
W20260709 13:17:18.793172 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55436.8us
I20260709 13:17:18.797675 11398 raft_consensus.cc:1275] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:18.880713 11548 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:17:18.899338 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 91718.5us
W20260709 13:17:18.935323 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 128277us
W20260709 13:17:18.935948 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 123041us
W20260709 13:17:18.944622 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89195.1us
I20260709 13:17:18.957816 16818 consensus_queue.cc:1048] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:18.958827 11548 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:19.010639 11294 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 7 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:19.013075 11294 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.025930 16818 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:19.045650 11298 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 218) took 19669 ms (client timeout 15000 ms). Trace:
I20260709 13:17:19.029968 16828 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:17:19.046113 11298 rpcz_store.cc:269] 0709 13:16:59.375947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.376165 (+   218us) service_pool.cc:224] Handling call
0709 13:17:19.045574 (+19669409us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:18.871150 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:18.871710 (+   560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:18.871746 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:17:18.871762 (+    16us) tablet.cc:662] Decoding operations
0709 13:17:18.872445 (+   683us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:18.872497 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:18.872523 (+    26us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:17:18.873019 (+   496us) tablet.cc:701] Row locks acquired
0709 13:17:18.873046 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:17:18.873199 (+   153us) write_op.cc:270] Start()
0709 13:17:18.873311 (+   112us) write_op.cc:276] Timestamp: P: 1783603038873142 usec, L: 0
0709 13:17:18.873332 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:18.880906 (+  7574us) log.cc:844] Serialized 2024 byte log entry
0709 13:17:18.892258 (+ 11352us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.894735 (+  2477us) write_op.cc:301] APPLY: starting
0709 13:17:18.894849 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.897505 (+  2656us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.897604 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:18.909941 (+ 12337us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:18.910113 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:17:18.910155 (+    42us) write_op.cc:312] APPLY: finished
0709 13:17:18.910394 (+   239us) log.cc:844] Serialized 57 byte log entry
0709 13:17:18.911190 (+   796us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.913793 (+  2603us) write_op.cc:454] Released schema lock
0709 13:17:18.913848 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:18.927536 (+ 13688us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.927773 (+   237us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3868,"reported_tablets":51,"spinlock_wait_cycles":469760,"child_traces":[["op",{"apply.queue_time_us":2219,"apply.run_cpu_time_us":7225,"apply.run_wall_time_us":32687,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":715,"prepare.run_cpu_time_us":3466,"prepare.run_wall_time_us":20293,"replication_time_us":18741}]]}
I20260709 13:17:19.159916 11395 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:19.168131 16824 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:19.170748 12112 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:19.258044 11548 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:19.265857 16799 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:19.284773 11398 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:19.288655 11293 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 8 to 9, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.293810 16799 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:19.353739 16813 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index 4 to 9, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.365173 11544 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 4 to 9, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.390679 11285 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 13:17:19.342206 11293 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 7 to 8, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 8 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.455193 11399 raft_consensus.cc:2955] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 4 to 9, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.519301 11297 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 10 to 11, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 11 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.575577 11297 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:19.590405 11297 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.595846 11297 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 4 to 6, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 6 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.638975 11293 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 232) took 7403 ms. Trace:
I20260709 13:17:19.639607 11293 rpcz_store.cc:276] 0709 13:17:12.235333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.235617 (+   284us) service_pool.cc:224] Handling call
0709 13:17:19.638874 (+7403257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:19.459989 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:19.460497 (+   508us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:19.460527 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:17:19.460544 (+    17us) tablet.cc:662] Decoding operations
0709 13:17:19.460923 (+   379us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:19.460971 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:19.461004 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:19.461296 (+   292us) tablet.cc:701] Row locks acquired
0709 13:17:19.461314 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:17:19.462145 (+   831us) write_op.cc:270] Start()
0709 13:17:19.462255 (+   110us) write_op.cc:276] Timestamp: P: 1783603039462097 usec, L: 0
0709 13:17:19.462276 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:19.463068 (+   792us) log.cc:844] Serialized 901 byte log entry
0709 13:17:19.466608 (+  3540us) op_driver.cc:464] REPLICATION: finished
0709 13:17:19.467009 (+   401us) write_op.cc:301] APPLY: starting
0709 13:17:19.467090 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:19.468367 (+  1277us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:19.468450 (+    83us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:19.469700 (+  1250us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:19.469860 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:17:19.469901 (+    41us) write_op.cc:312] APPLY: finished
0709 13:17:19.470085 (+   184us) log.cc:844] Serialized 33 byte log entry
0709 13:17:19.470228 (+   143us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.470303 (+    75us) write_op.cc:454] Released schema lock
0709 13:17:19.470344 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:19.470634 (+   290us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.470659 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1502108,"reported_tablets":51,"spinlock_wait_cycles":81152,"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":3614,"apply.run_wall_time_us":3613,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":1564,"prepare.run_cpu_time_us":2635,"prepare.run_wall_time_us":6532,"replication_time_us":4218,"spinlock_wait_cycles":4864}]]}
I20260709 13:17:19.667670 11294 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:17:19.669301 11294 catalog_manager.cc:5697] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.670941 11294 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 4 to 9, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:19.685029 11297 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 145) took 5307 ms. Trace:
I20260709 13:17:19.685468 11297 rpcz_store.cc:276] 0709 13:17:14.377256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.377473 (+   217us) service_pool.cc:224] Handling call
0709 13:17:14.468875 (+ 91402us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b5400002490. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f4b71 00007f8da67f50b2 00007f8daf76ea89 00007f8daf706da6 00007f8daf6d8088 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc
0709 13:17:14.574979 (+106104us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b5400002490. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f5640 00007f8daf6ccb76 00007f8daf7d6c3e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:19.684960 (+5109981us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:19.608406 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:19.609065 (+   659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:19.609109 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:17:19.609136 (+    27us) tablet.cc:662] Decoding operations
0709 13:17:19.609868 (+   732us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:19.609929 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:19.609964 (+    35us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:17:19.610418 (+   454us) tablet.cc:701] Row locks acquired
0709 13:17:19.610436 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:17:19.610596 (+   160us) write_op.cc:270] Start()
0709 13:17:19.610723 (+   127us) write_op.cc:276] Timestamp: P: 1783603039610548 usec, L: 0
0709 13:17:19.610751 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:17:19.611741 (+   990us) log.cc:844] Serialized 1650 byte log entry
0709 13:17:19.623357 (+ 11616us) op_driver.cc:464] REPLICATION: finished
0709 13:17:19.623805 (+   448us) write_op.cc:301] APPLY: starting
0709 13:17:19.623887 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:19.626236 (+  2349us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:19.626347 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:19.629767 (+  3420us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:19.641321 (+ 11554us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:17:19.641374 (+    53us) write_op.cc:312] APPLY: finished
0709 13:17:19.644485 (+  3111us) log.cc:844] Serialized 49 byte log entry
0709 13:17:19.645293 (+   808us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.645410 (+   117us) write_op.cc:454] Released schema lock
0709 13:17:19.645458 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:19.651216 (+  5758us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:19.651247 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":268779,"reported_tablets":27,"spinlock_wait_cycles":23417600,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":6402,"apply.run_wall_time_us":27380,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16728,"num_ops":4,"prepare.queue_time_us":155,"prepare.run_cpu_time_us":3531,"prepare.run_wall_time_us":6312,"replication_time_us":12453,"spinlock_wait_cycles":31488}]]}
W20260709 13:17:19.702241 11285 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 13474bcfb1f1443f82e90db3445bc880 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 9
I20260709 13:17:19.784063 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 13474bcfb1f1443f82e90db3445bc880 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:59884
W20260709 13:17:19.796034 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 61159.6us
W20260709 13:17:19.815994 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 92011.4us
I20260709 13:17:19.851831 16841 tablet_replica.cc:333] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:17:19.853407 16841 raft_consensus.cc:2243] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.854457 16841 raft_consensus.cc:2272] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.951300 16841 ts_tablet_manager.cc:1916] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:17:19.959175 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 149547us
I20260709 13:17:20.037865 11296 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 219) took 3625 ms. Trace:
I20260709 13:17:20.040835 11296 rpcz_store.cc:276] 0709 13:17:16.412475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.412685 (+   210us) service_pool.cc:224] Handling call
0709 13:17:20.037637 (+3624952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30045,"reported_tablets":51}
I20260709 13:17:20.059551 16841 ts_tablet_manager.cc:1929] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.8
I20260709 13:17:20.069324 16841 log.cc:1199] T 13474bcfb1f1443f82e90db3445bc880 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/13474bcfb1f1443f82e90db3445bc880
I20260709 13:17:20.071828 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 13474bcfb1f1443f82e90db3445bc880 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
W20260709 13:17:20.214102 11299 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 144) took 23749 ms (client timeout 15000 ms). Trace:
W20260709 13:17:20.217675 11299 rpcz_store.cc:269] 0709 13:16:56.441099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.441344 (+   245us) service_pool.cc:224] Handling call
0709 13:17:20.188717 (+23747373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42580,"reported_tablets":27}
I20260709 13:17:20.251296 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:20.403254 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:20.403996 11396 raft_consensus.cc:606] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:20.412820 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:20.413439 11399 raft_consensus.cc:606] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:20.416589 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:20.417140 11399 raft_consensus.cc:606] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:20.420450 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:20.421278 11399 raft_consensus.cc:606] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:20.424556 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:20.425405 11399 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:20.432171 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:20.434007 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:20.522564 16840 raft_consensus.cc:993] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:20.523360 16823 raft_consensus.cc:1081] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
W20260709 13:17:20.593502 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 73838.4us
I20260709 13:17:20.597651 16336 raft_consensus.cc:993] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:20.599200 16582 raft_consensus.cc:993] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:20.599741 16582 raft_consensus.cc:1081] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:20.600528 16582 raft_consensus.cc:1081] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:20.613873 11544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:20.614619 11544 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.615084 11544 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:20.672683 11547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:20.673388 11547 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.673740 11547 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:20.683434 11547 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.684670 11548 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:20.728487 11548 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.729194 11548 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:20.758906 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:20.759745 11396 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:20.760100 11396 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:20.765494 11396 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.766624 11396 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:20.769050 11547 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:20.776053 11548 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.785588 11544 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.794130 11396 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:17:20.818529 11544 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:20.820798 11548 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:20.824355 11516 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:20.825417 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:20.826057 11399 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:20.826333 11399 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:20.827389 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:20.827960 11396 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:20.828222 11396 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:20.837008 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70148.5us
I20260709 13:17:20.839830 11396 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.840871 11396 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:20.830946 11399 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:20.841632 16439 consensus_queue.cc:1175] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34fe1737ebb840fb9746980f649cc0a6" responder_term: 14 status { last_received { term: 14 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 14 index: 7 } }
I20260709 13:17:20.864606 11396 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:20.843112 16808 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:20.866353 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:20.866631 11399 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:20.883226 16175 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:17:20.908784 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 117964us
I20260709 13:17:20.917505 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:20.918165 11399 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:20.919919 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:20.933691 11475 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:20.948644 16175 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:20.969079 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:20.965855 16175 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:20.972070 11475 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:17:20.989370 11471 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:20.966437 16808 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:21.062856 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:17:21.070538 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85614.1us
I20260709 13:17:21.076617 11468 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:21.077379 11471 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
W20260709 13:17:21.078348 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 108899us
I20260709 13:17:21.064086 16858 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:17:21.098582 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 56951.3us
I20260709 13:17:21.101622 16858 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:21.111531 16808 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.171294 16858 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.190002 11468 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
W20260709 13:17:21.220752 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 121478us
W20260709 13:17:21.226127 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 306309us
I20260709 13:17:21.230281 16856 raft_consensus.cc:993] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:21.230923 16856 raft_consensus.cc:1081] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
W20260709 13:17:21.240955 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 139761us
I20260709 13:17:21.262050 16866 raft_consensus.cc:993] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:21.262676 16866 raft_consensus.cc:1081] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:21.271014 11293 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 14 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.293134 11548 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:21.293754 11548 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.294126 11548 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.303560 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:21.304239 11543 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.304606 11543 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:21.362313 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 139389us
I20260709 13:17:21.387208 11543 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.396956 11548 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.400204 11543 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:21.402362 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:21.406232 11468 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:21.415640 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:21.416569 11396 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:21.416956 11396 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:21.417960 11396 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.422525 11396 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:21.427412 11548 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:21.433506 11468 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:21.438232 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:21.439075 16858 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:21.439628 16858 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:21.440493 16858 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.444290 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:21.444944 11399 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:21.445246 11399 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:21.445974 11399 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.447753 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:21.452731 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:21.453337 11468 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.446911 11399 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.476425 11396 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:21.476612 11399 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:17:21.477232 11468 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:17:21.478322 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:21.479799 11543 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:21.481108 16175 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:21.481554 16175 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:21.482295 16175 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:21.506906 11468 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:21.516134 16873 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:21.575425 11468 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:21.606880 11396 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:21.608460 16858 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:21.631736 11471 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:21.633770 16175 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:17:21.635380 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 66658.9us
W20260709 13:17:21.645366 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 46637.2us
W20260709 13:17:21.738397 16884 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:17:21.745434 11476 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:21.747421 16864 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:21.812451 11399 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:21.838058 16175 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:21.864871 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.872893 11293 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 12 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.882500 11293 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 13 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.896991 11293 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:21.907387 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 64185.3us
W20260709 13:17:21.937417 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 99668.8us
I20260709 13:17:21.943755 11398 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:21.943755 11399 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:17:22.002161 11471 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:17:22.002405 16896 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:22.002203 16897 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:22.008874 16858 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:17:22.047704 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 123447us
I20260709 13:17:22.058666 11477 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:22.066362 16858 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:22.129943 11398 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:22.132002 16858 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:23.067981 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:23.241932 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66cb4598d33043fc8eb235647a242814"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:23.242596 11398 raft_consensus.cc:606] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:23.288239 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7abd40007d144575bd9e785e609a62df"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:23.289572 11398 raft_consensus.cc:606] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:23.293609 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:23.294296 11398 raft_consensus.cc:606] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:23.299367 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3fd80658464474dbe463b1943becc75"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:23.306044 11398 raft_consensus.cc:606] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:23.309712 11398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:23.316494 11398 raft_consensus.cc:606] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:23.317988 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:23.319808 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:23.513095 16439 raft_consensus.cc:993] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.513607 16439 raft_consensus.cc:1081] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.522961 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:23.523557 11543 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.523886 11543 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:23.535225 11543 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:23.541252 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:23.541855 11399 raft_consensus.cc:3055] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:17:23.542155 11399 raft_consensus.cc:740] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:23.542620 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:23.543300 12112 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:23.542867 11399 consensus_queue.cc:260] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:23.544145 11399 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:23.550845 11399 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:17:23.557116 12112 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:17:23.565413 11516 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:23.569548 16895 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:17:23.560205 11543 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:23.575829 16895 raft_consensus.cc:697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:23.576927 16895 consensus_queue.cc:237] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:23.628525 11296 catalog_manager.cc:5697] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 19 to 20, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 20 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.655292 16914 raft_consensus.cc:993] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.667965 16914 raft_consensus.cc:1081] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.666337 16863 raft_consensus.cc:993] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:23.669179 16863 raft_consensus.cc:1081] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:23.670806 11542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66cb4598d33043fc8eb235647a242814"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:23.671468 11542 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.671831 11542 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:23.670832 11477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7abd40007d144575bd9e785e609a62df"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:23.672606 11477 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.672941 11477 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:23.700318 16866 raft_consensus.cc:993] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.700898 16866 raft_consensus.cc:1081] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:23.702616 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3fd80658464474dbe463b1943becc75"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:23.703208 11551 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.703604 11551 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:23.729085 11477 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.757473 11477 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:23.722515 11551 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.759784 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:23.760490 11398 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:23.759008 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:23.760823 11398 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:23.761111 11548 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:23.761750 11398 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.772042 11398 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:23.768003 11551 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:23.769177 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:23.773656 11471 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:23.775156 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:23.775477 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:23.775813 11547 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:23.776042 11399 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:23.776338 11399 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:23.777020 11399 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.778100 11399 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:23.785470 11399 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:17:23.790325 11471 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:17:23.791298 11442 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:23.794116 11516 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:23.794740 16826 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:23.795331 16826 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:23.795035 16911 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:23.796247 16911 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:23.796221 16826 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.797367 16911 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.807504 16863 raft_consensus.cc:993] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:23.808089 16863 raft_consensus.cc:1081] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:23.809893 11471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:23.810575 11471 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.811050 11471 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:23.698505 11542 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.839488 11542 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:23.840456 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:23.841144 11477 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:23.847360 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:23.848057 11399 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:23.848351 11399 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:23.849100 11399 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.850247 11471 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.850575 11399 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:23.853545 11471 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:23.855581 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:23.856226 11396 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:23.856518 11396 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:23.857214 11396 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.855571 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:23.858753 11543 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:23.849026 11398 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:23.860132 11299 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.863777 11477 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:23.864953 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:23.865965 16913 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:23.866452 16913 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:23.867892 16913 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.875918 11396 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:23.877792 11543 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:23.879134 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:23.884117 11399 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:23.885319 16826 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:23.885878 16826 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:23.886793 16826 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:23.923698 11396 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:23.973474 11396 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:17:23.972826 11296 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.975332 16913 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:17:23.982097 11299 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.974532 11296 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 12 to 13, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:24.051932 12112 raft_consensus.cc:1275] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:17:24.053589 16910 consensus_queue.cc:1048] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:17:24.317998 11398 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:24.319533 16913 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:24.320832 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:24.348191 11399 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:24.352957 16826 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:24.369477 11477 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:24.371193 16911 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:24.377307 11543 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:24.379423 16826 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:24.385807 11399 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:24.390385 16907 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:24.421131 11468 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:24.427085 11398 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:24.428946 16910 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:24.440172 16910 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
W20260709 13:17:24.520287 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 75354us
W20260709 13:17:24.604844 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 80407.7us
W20260709 13:17:24.605262 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 63896.1us
I20260709 13:17:24.628638 11543 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:17:24.710443 16947 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:24.739022 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 134130us
W20260709 13:17:24.762514 16934 log.cc:927] Time spent T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
W20260709 13:17:24.887661 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 248785us
I20260709 13:17:25.503471 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:25.615653 11394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:25.616246 11394 raft_consensus.cc:606] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:25.619050 11394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59af340b26264692af5490466dd69b78"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:25.619663 11394 raft_consensus.cc:606] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:25.623258 11394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:25.623827 11394 raft_consensus.cc:606] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:25.624993 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:25.626641 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:25.680831 16866 raft_consensus.cc:993] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:25.681391 16954 raft_consensus.cc:1081] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:25.683171 11477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:25.683750 11477 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.684074 11477 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.705333 11477 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:25.709995 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:25.710625 11394 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:25.710917 11394 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:25.711833 11477 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:25.711715 11394 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:25.713092 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:25.715247 12123 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.719305 11394 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.734972 12123 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:25.737102 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:25.738070 16947 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:25.738587 16947 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:25.739534 16947 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:25.743317 11394 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:25.788522 11296 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 14 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 15 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.859522 16866 raft_consensus.cc:993] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:25.860222 16971 raft_consensus.cc:1081] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:25.865408 16954 raft_consensus.cc:993] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:25.865890 16954 raft_consensus.cc:1081] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:25.868937 11548 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:25.869526 11548 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.869887 11548 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.888664 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59af340b26264692af5490466dd69b78"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:25.889333 11543 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.889735 11543 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:25.927590 11548 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:25.936487 11548 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:25.937621 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:25.940282 11396 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:25.940635 11396 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:25.941296 11396 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:25.938035 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:25.936949 11543 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:25.942338 11477 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.945235 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:25.945823 12123 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:25.946197 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:25.946724 11399 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:17:25.947006 11399 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:25.947734 11399 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:25.948762 11399 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:25.956385 11543 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:25.963155 12123 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:17:25.963610 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:17:25.968609 11477 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:25.964936 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:25.970112 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:25.973266 11396 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:25.975312 16909 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:25.975873 16909 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:25.983376 11396 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:25.977039 16911 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:17:26.008946 16911 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:25.991322 16909 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:26.010058 16911 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:26.058832 11293 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 14 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.167213 11299 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 19 to 20, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 20 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:26.181854 12123 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:26.183451 16940 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:26.277343 11399 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:26.279109 16947 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:26.491539 12118 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:17:26.494244 16970 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:26.569681 11394 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:17:26.571367 16911 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:26.579407 11477 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:26.584730 16911 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:26.603302 11399 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:26.604812 16911 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:26.627609 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:28.232770 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:28.569707 11471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:28.570758 11471 raft_consensus.cc:606] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:28.615906 11543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:28.616554 11543 raft_consensus.cc:606] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:28.618206 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:28.619881 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:28.931646 16995 raft_consensus.cc:993] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:28.933688 16995 raft_consensus.cc:1081] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:28.949980 11399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:28.950721 11399 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:28.951648 11399 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:28.966138 11399 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:28.970335 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:28.971222 11475 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:28.971549 11475 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:28.972364 11475 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:28.973591 11475 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:28.970379 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:28.974951 11543 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:28.975703 11399 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:28.989158 11543 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:28.991168 11367 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:28.992205 16996 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:28.992681 16996 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:28.993568 16996 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:28.989158 11475 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:29.016314 16994 raft_consensus.cc:993] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:29.031020 16994 raft_consensus.cc:1081] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:29.035185 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:17:29.036345 11475 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.041575 11299 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.045694 11475 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:29.065837 11475 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:29.096361 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:29.096972 11399 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:29.097308 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:29.097913 11551 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:29.098205 11551 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:29.099061 11551 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:29.100181 11551 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:29.103139 11475 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:29.108909 11399 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:29.110801 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:29.110900 11551 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:17:29.112083 17003 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:29.112596 17003 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:29.113541 17003 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:29.272482 11299 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 14 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 15 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.491984 11477 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:29.499294 16964 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:29.548266 11398 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:29.551486 11551 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:29.551617 17003 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:29.553357 16996 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:29.598766 11548 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:29.600726 16995 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:29.624019 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:29.688524 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44345.6us
W20260709 13:17:30.881517 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 109380us
I20260709 13:17:30.887976 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:31.067051 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:31.067986 11395 raft_consensus.cc:606] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:31.071326 11548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:31.071919 11548 raft_consensus.cc:606] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:31.075531 11475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7abd40007d144575bd9e785e609a62df"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:31.076078 11475 raft_consensus.cc:606] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:31.078037 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:31.079743 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:31.156347 16940 raft_consensus.cc:993] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:31.156816 16940 raft_consensus.cc:1081] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:31.165004 11395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7abd40007d144575bd9e785e609a62df"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:31.165587 11395 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:31.165915 11395 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:31.181872 16994 raft_consensus.cc:993] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:31.182475 16994 raft_consensus.cc:1081] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:31.186069 11395 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.187656 11394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41730
I20260709 13:17:31.213524 11394 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:31.213901 11394 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:31.223812 11394 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:31.231173 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:31.231814 11543 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:31.232121 11543 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:31.232832 11543 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:31.233915 11543 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:31.235026 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:31.235586 11394 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:31.235680 11477 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:31.237759 11477 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:31.238545 11477 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.241068 11477 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:31.235343 11395 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:31.236527 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:31.250824 12113 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:31.236755 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:31.256405 11548 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:31.257028 11543 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:31.259546 11367 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:31.260556 17021 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:31.261075 17021 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:31.261965 17021 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:31.269004 11548 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:31.276870 11367 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:31.277967 17029 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:31.273761 11477 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 13.
I20260709 13:17:31.274216 12113 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:31.278482 17029 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:31.280094 17029 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.312808 11294 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.462299 11294 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 12 to 13, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.576964 17029 raft_consensus.cc:993] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:31.577505 17029 raft_consensus.cc:1081] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:31.579766 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:31.580322 11543 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:31.580655 11543 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:31.600361 11543 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.608351 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:31.608973 11394 raft_consensus.cc:3055] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:31.609257 11394 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:31.609959 11394 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.610940 11394 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:31.626649 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:31.627388 11471 raft_consensus.cc:3060] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:31.646962 11543 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:31.648422 11471 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:17:31.652204 11514 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:31.653353 17024 raft_consensus.cc:2804] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:31.653893 17024 raft_consensus.cc:697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:31.654806 17024 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:31.649194 11394 raft_consensus.cc:2468] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:17:31.687434 11294 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 17 to 18, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:31.712390 11471 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:31.714771 16996 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:31.747346 11543 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:31.749580 17021 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:31.771874 12113 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:31.777868 17029 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:31.794853 11548 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:31.800922 17021 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:32.086022 11396 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:17:32.088140 17018 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:32.100960 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:32.153103 11477 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:17:32.157027 17024 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:17:32.673262 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 54052.7us
I20260709 13:17:33.260643 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:33.773387 17021 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:33.774139 17021 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.776455 17021 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.834259 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.835336 17060 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:33.835947 17060 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.838143 17060 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.845685 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.883025 17055 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:33.883785 17055 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.886385 17055 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:33.890843 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 63267.2us
I20260709 13:17:33.901964 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:33.903234 17003 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:33.903904 17003 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.905997 17003 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.906091 17018 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:33.906718 17018 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:33.908161 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08000a3cc0 after lost signal to thread 10255
W20260709 13:17:33.909349 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08001fac80 after lost signal to thread 11334
I20260709 13:17:33.914008 17048 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:33.915712 17002 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:33.914592 17048 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:33.916280 17002 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:33.918794 17002 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.919814 17048 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:33.935276 11331 debug-util.cc:398] Leaking SignalData structure 0x7b0800188d20 after lost signal to thread 11426
I20260709 13:17:33.939181 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:33.939561 11331 debug-util.cc:398] Leaking SignalData structure 0x7b0800188cc0 after lost signal to thread 11501
I20260709 13:17:33.918861 17018 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:17:33.941308 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08003f84a0 after lost signal to thread 11575
I20260709 13:17:33.941569 17044 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:33.942152 17044 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:33.942641 11331 debug-util.cc:398] Leaking SignalData structure 0x7b08002a96a0 after lost signal to thread 12147
I20260709 13:17:33.944393 17044 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:33.946463 11551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:33.949597 11551 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:33.960695 17021 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:33.961349 17021 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.962431 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:33.963374 17021 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.964699 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:33.966063 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.973774 17052 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:33.974526 17052 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:33.976745 17052 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:33.980080 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.981801 17052 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:33.982378 17052 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:33.983577 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79107.8us
I20260709 13:17:33.984341 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.988880 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:33.991526 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.004285 17052 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.005151 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.027945 17050 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:34.028529 17050 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.030778 17021 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:34.031311 17021 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.032943 17021 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.034585 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.035755 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.039160 17029 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:34.039883 17029 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:34.040370 17029 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.042523 17029 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.043203 17050 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.044454 17072 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:34.045670 17068 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:34.046259 17068 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:34.047653 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 146965us
I20260709 13:17:34.048465 17068 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.051442 17056 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:34.051977 17056 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.053983 17056 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.054421 17056 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.055903 17072 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:34.056537 17072 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.054927 17056 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.060696 17056 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.061491 17072 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.076237 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.081367 17045 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.081528 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.082039 17045 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.083097 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.085106 17045 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:34.086114 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.086290 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.087299 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.088754 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.063231 17069 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.105317 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.106297 11395 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:17:34.112897 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:34.115953 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:34.116524 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.117205 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.117355 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:17:34.117496 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 81818.3us
I20260709 13:17:34.121719 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.124471 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:34.124730 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.125102 11399 raft_consensus.cc:606] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.125440 11548 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:34.113840 17069 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.128540 17069 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:34.129966 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.132714 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.132615 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.133729 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.134756 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.138285 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.139977 11367 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.144351 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:34.145612 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:34.147593 17073 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.148293 17073 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.149163 17021 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:34.152261 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.154672 17073 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.155270 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.156170 17073 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:34.157307 17073 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.157792 17073 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:34.163069 11399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:34.163676 11399 raft_consensus.cc:606] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.165061 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:34.166716 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:34.167479 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.173638 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.176944 11367 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.178823 17074 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:34.179111 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.180879 17074 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:34.184551 11367 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.185786 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.186590 17068 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:34.187251 17068 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:34.196362 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:34.202334 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:34.203227 17074 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:34.204402 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:34.206218 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:34.206930 17068 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:34.207748 17068 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:34.209213 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.209968 17072 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:34.214156 17073 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.219841 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.221412 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.229461 17069 raft_consensus.cc:993] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:34.229970 17069 raft_consensus.cc:1081] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:34.239703 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:34.242686 11516 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.248350 17064 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:34.248821 17064 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.248873 11516 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.249254 17064 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:34.250079 17044 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:34.251538 11516 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.252729 17045 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:34.254055 11516 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.254909 17045 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:34.257275 17064 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.258265 17072 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:34.259109 11395 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:34.261376 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:34.271974 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.273813 16995 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:34.275976 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.276939 17073 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.277112 17048 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:34.278882 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:34.279145 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.280249 17048 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:34.284927 11367 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:34.287501 11548 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:34.289888 17072 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:34.292673 17059 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:34.293479 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:34.293638 11393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41730
I20260709 13:17:34.294068 17070 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:34.294265 11393 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:34.294471 17070 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:34.294660 11393 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.295614 17070 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.300212 17070 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:34.308413 11393 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.315083 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:34.315930 17059 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:34.316354 17045 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:34.317186 16995 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:34.317571 16995 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:34.318455 16995 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.318886 17064 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:34.320525 16995 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:34.320438 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:34.322027 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:34.331529 11470 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.332901 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:34.323305 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:34.335255 11548 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:34.335609 11548 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:34.336377 11548 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.337585 11548 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:34.345832 11470 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:34.325678 11393 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.352262 11365 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:34.355479 17072 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:34.356036 17072 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:34.356994 17072 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.360260 11548 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:34.362869 11477 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:17:34.370391 11514 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:34.371690 17044 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:34.372195 17044 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:34.374235 17044 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.384302 17074 raft_consensus.cc:993] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:34.384820 17074 raft_consensus.cc:1081] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:34.386858 11545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:34.387408 11545 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:34.387760 11545 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:34.392526 17058 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:34.393440 17058 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.397501 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:34.411164 11296 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 12 to 13, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:34.415434 11545 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.417362 11545 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.419639 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:34.420243 11396 raft_consensus.cc:3055] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:34.420531 11396 raft_consensus.cc:740] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:34.421248 11396 consensus_queue.cc:260] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.421981 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:34.422446 11396 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:34.422533 12118 raft_consensus.cc:3060] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:34.430540 11294 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 11, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.397316 17058 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:34.460281 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:34.467146 12118 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:34.469058 17056 raft_consensus.cc:993] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:34.469928 17056 raft_consensus.cc:1081] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:34.471863 11396 raft_consensus.cc:2468] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:34.474956 11477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:34.477641 11477 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:34.478084 11477 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:34.486331 11477 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.490059 11516 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:34.491065 17069 raft_consensus.cc:2804] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:34.492372 17069 raft_consensus.cc:697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:34.493281 17069 consensus_queue.cc:237] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:34.498463 11477 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:34.505038 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:34.507819 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:34.515712 11393 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:34.516041 11393 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:34.516216 17044 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:34.516832 11393 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.517939 11393 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:34.509670 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:34.519520 11545 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:34.532735 11545 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:34.525729 11393 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:34.539958 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:34.541193 17070 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:34.541719 17070 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:34.542657 17070 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:34.611236 11299 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.632810 11294 catalog_manager.cc:5697] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 14 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.756385 11477 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:34.758188 17074 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:34.764268 11393 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:34.791219 11393 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:34.857203 17045 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:34.935513 12118 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:34.939338 11475 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:34.939168 17044 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:34.947597 17045 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:17:35.028542 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 87426.5us
I20260709 13:17:35.032555 11545 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:17:35.055717 17068 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:17:35.110150 11398 raft_consensus.cc:1275] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:35.114804 11545 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:35.119786 17066 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:35.124127 17069 consensus_queue.cc:1048] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:35.175365 11396 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:35.185910 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:35.200398 17066 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 13:17:35.325208 17092 log.cc:927] Time spent T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:17:35.417336 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 113512us
I20260709 13:17:35.570469 11476 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:17:35.645835 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 168816us
W20260709 13:17:35.657675 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 152097us
W20260709 13:17:35.663738 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 148528us
I20260709 13:17:35.739355 11396 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:17:35.769493 17128 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:35.777045 17120 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:35.795024 17130 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:35.795735 17130 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:35.798314 17130 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:35.816155 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:35.817029 11477 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:35.822955 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:35.824602 17045 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:35.825032 17045 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:35.825407 17045 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:35.838358 17045 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:35.860133 12113 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:17:35.865131 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 355355us
I20260709 13:17:35.892019 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:35.892822 11399 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:17:35.896550 17120 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:35.933939 17045 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:35.937987 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:35.931736 17109 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 3 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:35.943256 11399 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 3 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:35.954264 11475 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:17:35.955335 17056 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.956007 17056 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:35.959255 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:35.960147 11477 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:35.965067 17056 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:35.969046 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:35.970002 11396 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 15: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
W20260709 13:17:35.971216 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50227.1us
I20260709 13:17:35.977281 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:35.979291 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:35.980031 11542 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:35.990267 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:35.991232 17130 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:35.991909 17130 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:35.993007 17130 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.006371 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b61ac66d97f84c1186c6e2b1d79477b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:17:36.014880 11294 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 3 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:36.017956 17072 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:36.019750 17072 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.020136 17072 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:36.056528 17072 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.063887 11296 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 14 to 15, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:36.073392 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:36.074222 11475 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:36.078400 12113 raft_consensus.cc:2955] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 3 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:36.079583 17072 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:36.080895 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:36.083124 11475 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:36.084275 11365 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:36.085448 17072 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:36.086253 17072 raft_consensus.cc:697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:36.087256 17072 consensus_queue.cc:237] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.295924 11299 catalog_manager.cc:5697] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 16, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 16 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:36.331691 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 45182.8us
I20260709 13:17:36.347667 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet b61ac66d97f84c1186c6e2b1d79477b2 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:51266
I20260709 13:17:36.364228 17145 tablet_replica.cc:333] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:17:36.367395 17145 raft_consensus.cc:2243] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.373730 17145 raft_consensus.cc:2272] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.393711 17145 ts_tablet_manager.cc:1916] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:36.448391 17145 ts_tablet_manager.cc:1929] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.11
I20260709 13:17:36.450383 17145 log.cc:1199] T b61ac66d97f84c1186c6e2b1d79477b2 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/b61ac66d97f84c1186c6e2b1d79477b2
I20260709 13:17:36.451035 11402 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:36.452481 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet b61ac66d97f84c1186c6e2b1d79477b2 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:36.452716 17132 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:36.453495 17132 consensus_queue.cc:1243] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer 147201d22a694ca59f6e014f083a0892 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:36.453986 17132 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:36.454300 17132 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:36.455134 17132 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.459388 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:36.483213 17132 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:36.569188 11471 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:36.570807 17021 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:36.682092 11548 raft_consensus.cc:1275] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:17:36.682548 11548 pending_rounds.cc:85] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 8
I20260709 13:17:36.682796 11548 pending_rounds.cc:107] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:17:36.683141 11548 raft_consensus.cc:2889] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:36.687683 17021 consensus_queue.cc:1048] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:36.707870 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:36.708762 11402 raft_consensus.cc:606] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:36.712013 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66cb4598d33043fc8eb235647a242814"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:36.712550 11542 raft_consensus.cc:606] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:36.717448 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930d965be7eb49448fa23456beb9925a"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:36.717993 11542 raft_consensus.cc:606] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:36.724551 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:36.725116 11402 raft_consensus.cc:606] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:36.735311 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:17:36.737660 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:36.749107 17132 raft_consensus.cc:993] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:36.749818 17112 raft_consensus.cc:1081] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:36.751590 11396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "930d965be7eb49448fa23456beb9925a"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41730
I20260709 13:17:36.752164 11396 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:36.752522 11396 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:36.772272 11396 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.777730 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:36.778375 11477 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:36.786737 11477 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:36.790150 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:36.790640 11365 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:36.790836 11548 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:36.791249 11548 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:36.791604 17072 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:36.792178 17072 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:36.792081 11548 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.793366 11548 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:36.793129 17072 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:36.802928 11396 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:36.811674 11548 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:36.821651 11299 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 14 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.227432 17112 raft_consensus.cc:993] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.228646 17155 raft_consensus.cc:1081] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.234949 11477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66cb4598d33043fc8eb235647a242814"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:17:37.235970 11477 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:37.236465 11477 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:37.303097 17156 raft_consensus.cc:993] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.304648 17021 raft_consensus.cc:1081] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.305310 17116 raft_consensus.cc:993] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.305927 17116 raft_consensus.cc:1081] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:37.334808 11473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:37.334964 11477 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.343082 11471 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:37.335563 11473 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:37.344612 11473 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:37.336177 11470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:37.346783 11470 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:37.347234 11470 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.354934 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:37.355675 11396 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:37.357178 17156 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:37.345935 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:37.343900 11477 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:37.359797 11548 raft_consensus.cc:3055] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:37.360422 11548 raft_consensus.cc:740] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:37.368476 11548 consensus_queue.cc:260] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.370932 11548 raft_consensus.cc:3060] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:37.383721 11396 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:37.394455 11442 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:37.401542 11470 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.409718 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:37.409988 17120 raft_consensus.cc:2804] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:37.410387 11396 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:37.410692 17120 raft_consensus.cc:697] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:37.410792 11396 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:37.411909 17120 consensus_queue.cc:237] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.412581 11396 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.414165 11396 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.423564 11396 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:37.423892 11473 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.434160 11551 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:37.436813 17158 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:37.438287 11470 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:37.457350 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:37.462037 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:37.469614 11549 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:37.475239 17159 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:37.476111 17159 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:37.477392 17159 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.480389 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:37.481227 11396 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:37.481613 11396 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:37.489137 11473 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:37.482879 11396 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:37.498256 11396 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:37.513844 11548 raft_consensus.cc:2468] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:17:37.515859 11549 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:17:37.521036 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:37.523397 11545 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:37.526505 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 76847.7us
I20260709 13:17:37.530675 11396 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:37.550364 11299 catalog_manager.cc:5697] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.569346 11545 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:37.592567 11442 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:37.608310 17160 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:37.608954 17160 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:37.610033 17160 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:37.655831 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 43855.2us
I20260709 13:17:37.771250 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:37.822392 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 106950us
W20260709 13:17:37.823896 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 117541us
W20260709 13:17:37.832772 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 64538.4us
I20260709 13:17:37.894042 11549 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:37.898411 17174 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:37.915572 11399 raft_consensus.cc:1275] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:37.926556 17176 consensus_queue.cc:1048] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:37.959332 11299 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 9 to 10, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 10 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.961637 11299 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.051774 11402 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:38.064772 17120 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:38.087527 11396 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:38.087862 11551 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:17:38.098443 17120 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:38.102634 17120 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:38.187916 11549 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:38.199199 17160 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:17:38.217742 17184 log.cc:927] Time spent T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.052s	user 0.004s	sys 0.000s
I20260709 13:17:38.255986 11551 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:17:38.319047 11515 consensus_peers.cc:597] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a6ebf03c40d4f1fa404e0fe9f8f9fc5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.320611 11396 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:38.321274 11473 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:38.322417 17173 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:38.324323 17173 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:38.336421 17155 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.338243 11396 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.369347 11549 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:38.377416 11473 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.377981 11299 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
W20260709 13:17:38.427727 11515 consensus_peers.cc:597] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2eeda82ae3942d6bf0db873fa350bdc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.453924 11402 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:38.482080 11473 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:38.487505 17155 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:38.547093 17155 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:38.584703 17155 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.635138 11477 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.636713 11395 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.652083 11299 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.750747 11477 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:38.761977 11551 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:17:38.762918 11441 consensus_peers.cc:597] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8eacd98a9dc4961a9fac3d2ae9dbd02. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.764621 11395 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:38.767359 17174 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:38.787103 17120 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:38.806396 17174 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.807587 11549 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.829258 11395 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.852061 11297 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.863339 17196 ts_tablet_manager.cc:933] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:38.909281 11477 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:38.929692 17198 ts_tablet_manager.cc:933] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:38.937116 11551 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:38.937523 11399 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:17:38.938613 11441 consensus_peers.cc:597] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08ebf4312af74188830a07342d0dc119. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.940903 17120 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:38.943081 17120 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:38.964391 17198 tablet_copy_client.cc:323] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:38.966851 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet b2eeda82ae3942d6bf0db873fa350bdc from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:38.967514 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet b2eeda82ae3942d6bf0db873fa350bdc from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-b2eeda82ae3942d6bf0db873fa350bdc
I20260709 13:17:38.971437 11549 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.968855 17176 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.981810 11561 tablet_copy_source_session.cc:215] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:38.987385 17196 tablet_copy_client.cc:323] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:38.989647 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:38.990231 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-6a6ebf03c40d4f1fa404e0fe9f8f9fc5
I20260709 13:17:38.994336 11297 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:38.995043 17198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2eeda82ae3942d6bf0db873fa350bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.999389 11561 tablet_copy_source_session.cc:215] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:39.007488 17196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:39.015843 11477 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:39.016532 11477 raft_consensus.cc:606] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership
I20260709 13:17:39.021663 11477 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:39.022292 11477 raft_consensus.cc:606] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership
W20260709 13:17:39.025394 11284 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for b8eacd98a9dc4961a9fac3d2ae9dbd02 already in progress [suppressed 18 similar messages]
I20260709 13:17:39.042567 11395 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:39.048099 17198 tablet_copy_client.cc:806] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.058264 17198 tablet_copy_client.cc:670] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.062881 17198 tablet_copy_client.cc:538] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.074807 17198 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:39.079998 11477 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:39.080734 11477 raft_consensus.cc:606] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership
W20260709 13:17:39.090137 11348 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:17:39.090574 11348 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for b8eacd98a9dc4961a9fac3d2ae9dbd02 already in progress
W20260709 13:17:39.090857 11348 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for b8eacd98a9dc4961a9fac3d2ae9dbd02 already in progress
I20260709 13:17:39.097714 17196 tablet_copy_client.cc:806] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.117935 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:39.130585 17196 tablet_copy_client.cc:670] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.139707 17196 tablet_copy_client.cc:538] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.254413 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:39.261495 11473 raft_consensus.cc:606] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:39.267308 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:39.267956 11395 raft_consensus.cc:606] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:39.277170 17196 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:39.277894 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:39.305277 11473 raft_consensus.cc:606] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:39.307180 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:39.309101 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:39.310334 17174 raft_consensus.cc:993] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.310846 17174 raft_consensus.cc:1081] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.312595 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:17:39.313109 11551 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.313450 11551 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:39.292680 17198 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.288865 17205 ts_tablet_manager.cc:933] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:39.325241 17198 tablet_bootstrap.cc:492] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:39.336931 11551 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.339599 17198 ts_tablet_manager.cc:1403] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.265s	user 0.033s	sys 0.025s
I20260709 13:17:39.341768 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:39.342635 11402 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:39.343828 11551 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:39.348143 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:39.351527 11473 raft_consensus.cc:3055] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:39.351912 11473 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:39.352883 11473 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.355207 11473 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:39.374514 11402 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:17:39.376870 11516 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:39.378845 17173 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:39.379395 17173 raft_consensus.cc:697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:39.380342 17173 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.391494 17198 raft_consensus.cc:359] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.392381 17198 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:39.393013 17198 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.399096 11473 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:17:39.397527 11299 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.399767 17205 tablet_copy_client.cc:323] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:17:39.405400 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:17:39.406050 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-b8eacd98a9dc4961a9fac3d2ae9dbd02
I20260709 13:17:39.430523 17198 ts_tablet_manager.cc:1434] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.090s	user 0.004s	sys 0.003s
I20260709 13:17:39.414783 11487 tablet_copy_source_session.cc:215] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:39.432441 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b2eeda82ae3942d6bf0db873fa350bdc received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:39.432958 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b2eeda82ae3942d6bf0db873fa350bdc on tablet b2eeda82ae3942d6bf0db873fa350bdc with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:39.442181 17205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:39.521163 17196 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.522003 17205 tablet_copy_client.cc:806] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.522255 17196 tablet_bootstrap.cc:492] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:39.531805 17198 ts_tablet_manager.cc:933] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:39.535449 17205 tablet_copy_client.cc:670] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.539024 17196 ts_tablet_manager.cc:1403] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.262s	user 0.033s	sys 0.013s
I20260709 13:17:39.556463 17205 tablet_copy_client.cc:538] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.561278 17196 raft_consensus.cc:359] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.562201 17196 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:39.564169 17196 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:39.570151 17196 ts_tablet_manager.cc:1434] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:17:39.571461 17205 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:39.572840 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6a6ebf03c40d4f1fa404e0fe9f8f9fc5 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:39.573338 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-6a6ebf03c40d4f1fa404e0fe9f8f9fc5 on tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:39.590307 17198 tablet_copy_client.cc:323] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:17:39.592103 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 08ebf4312af74188830a07342d0dc119 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:17:39.592679 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 08ebf4312af74188830a07342d0dc119 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-08ebf4312af74188830a07342d0dc119
I20260709 13:17:39.613706 11487 tablet_copy_source_session.cc:215] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:39.628141 17198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ebf4312af74188830a07342d0dc119. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:39.696558 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44610.1us
I20260709 13:17:39.708614 17156 raft_consensus.cc:993] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.709131 17156 raft_consensus.cc:1081] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.717779 11545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:39.718356 11545 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.718737 11545 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:39.719524 17174 raft_consensus.cc:993] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:39.720139 17131 raft_consensus.cc:1081] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:39.724520 11402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b22c9e294a846159d4831b92f692943"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:39.725050 11402 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.725402 11402 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:39.727950 11545 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:39.737558 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:39.737869 17198 tablet_copy_client.cc:806] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.738216 11395 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:39.738435 17198 tablet_copy_client.cc:670] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.738551 11395 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:39.735917 11545 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:39.739437 11395 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:39.740470 11395 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:39.748788 17198 tablet_copy_client.cc:538] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.751394 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:39.752298 11477 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:39.776754 11473 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:39.780582 11396 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:39.785825 17194 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:39.789652 12113 raft_consensus.cc:1217] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.10->[13.11-13.11]   Dedup: 13.11->[]
I20260709 13:17:39.794374 17205 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.797283 17173 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:39.806243 17174 raft_consensus.cc:993] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.808975 17174 raft_consensus.cc:1081] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:39.849562 11477 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:17:39.864737 11395 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 9.
I20260709 13:17:39.864416 11402 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:39.855657 17205 tablet_bootstrap.cc:492] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
W20260709 13:17:39.884503 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 78744.7us
W20260709 13:17:39.891630 11515 consensus_peers.cc:597] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:39.899712 11551 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:17:39.900369 11551 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.900741 11551 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:39.902444 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:39.903035 11477 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:39.903309 11477 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:39.893231 17205 ts_tablet_manager.cc:1403] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.322s	user 0.040s	sys 0.009s
I20260709 13:17:39.903919 11477 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:39.910869 11477 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:39.914227 11402 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:39.936295 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46560.9us
I20260709 13:17:39.937290 11514 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:39.946391 17173 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:39.947005 17173 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:39.948033 17173 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:39.954427 12113 raft_consensus.cc:1217] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 13:17:39.962013 11477 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
W20260709 13:17:39.965220 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71986us
W20260709 13:17:40.002756 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65539.1us
I20260709 13:17:40.031855 17198 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:40.040185 11551 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:40.069955 11365 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:40.071508 11551 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:40.075815 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:40.076504 11399 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:40.079046 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 115174us
W20260709 13:17:40.092989 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 113882us
I20260709 13:17:40.099206 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:40.099999 11545 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:40.084204 17209 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:40.118353 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:40.122814 11471 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:40.123185 11471 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:40.134192 11471 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:40.132467 17209 raft_consensus.cc:697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:40.172065 17205 raft_consensus.cc:359] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:40.173194 17205 raft_consensus.cc:740] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:40.174023 17205 consensus_queue.cc:260] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:40.176394 11545 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:17:40.182304 11399 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:40.199119 17209 consensus_queue.cc:237] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:40.215489 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 94358.3us
W20260709 13:17:40.230438 17228 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.199s	user 0.003s	sys 0.000s
I20260709 13:17:40.231714 17205 ts_tablet_manager.cc:1434] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.319s	user 0.009s	sys 0.000s
I20260709 13:17:40.302884 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:40.304797 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b8eacd98a9dc4961a9fac3d2ae9dbd02 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:17:40.305397 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b8eacd98a9dc4961a9fac3d2ae9dbd02 on tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:40.310137 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:40.334875 17234 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:40.335587 17234 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:40.336527 17234 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:40.401046 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 58781.7us
W20260709 13:17:40.414785 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 63230.2us
W20260709 13:17:40.495316 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 154880us
W20260709 13:17:40.497074 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 157379us
I20260709 13:17:40.522137 11471 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:40.544332 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 104575us
W20260709 13:17:40.552488 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44699us
W20260709 13:17:40.552440 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 139882us
I20260709 13:17:40.566666 11399 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14. Ops: []
W20260709 13:17:40.589649 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.631270 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 341865us
I20260709 13:17:40.662436 11399 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:40.671061 11471 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:17:40.679287 17222 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:40.675845 11299 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 8 to 9, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:40.723706 11548 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:40.710340 11296 catalog_manager.cc:5697] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 10 to 11, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:40.746621 17225 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:40.747475 11476 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:40.787357 17224 raft_consensus.cc:1064] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:40.792918 17223 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:40.793509 17249 raft_consensus.cc:1064] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:40.796377 17249 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:40.819469 11477 raft_consensus.cc:1275] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:40.824983 17021 consensus_queue.cc:1048] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:40.809535 17224 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:40.952739 17223 raft_consensus.cc:1064] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:40.959213 11402 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:40.963634 11396 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:40.964788 17250 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:40.977197 12113 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:40.979696 17234 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:40.985394 11470 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:17:40.987223 11470 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:40.989657 17234 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:17:40.991564 17198 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:17:41.000528 17198 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=2 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:17:41.015307 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50679.1us
I20260709 13:17:41.055568 17198 tablet_bootstrap.cc:492] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:41.056404 17198 ts_tablet_manager.cc:1403] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 1.025s	user 0.052s	sys 0.021s
I20260709 13:17:41.000864 17223 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 13, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:41.019984 17250 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:17:41.029451 17249 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:17:41.110556 12112 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:41.115295 17198 raft_consensus.cc:359] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:41.116099 17198 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:41.116117 11473 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:41.119314 17198 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:41.112737 17222 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:17:41.134001 11399 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:41.133198 17250 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:17:41.157403 12112 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:17:41.164422 17198 ts_tablet_manager.cc:1434] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.000s
I20260709 13:17:41.174695 11476 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:17:41.178416 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 86399.3us
I20260709 13:17:41.181432 11399 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:17:41.189538 17249 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:17:41.175457 17271 log.cc:927] Time spent T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:17:41.200034 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-08ebf4312af74188830a07342d0dc119 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:17:41.228554 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-08ebf4312af74188830a07342d0dc119 on tablet 08ebf4312af74188830a07342d0dc119 with peer b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:41.230234 17272 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:17:41.179499 17231 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:17:41.243232 17240 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:17:41.248486 17190 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.268906 11471 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.252095 17215 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
W20260709 13:17:41.338006 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 92294.1us
I20260709 13:17:41.339352 12112 raft_consensus.cc:1217] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.8->[13.9-13.9]   Dedup: 13.9->[]
I20260709 13:17:41.351258 11395 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:41.480397 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 131206us
I20260709 13:17:41.492440 12119 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:41.520803 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 44314.9us
W20260709 13:17:41.545050 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 196966us
I20260709 13:17:41.555945 17221 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.563176 17215 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Committing config change with OpId 17.15: config changed from index 13 to 15, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.575861 11297 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.579603 11402 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 13 to 15, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.585417 12122 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 13 to 15, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.578390 11297 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 13 to 14, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.608557 11394 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:41.619606 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 60406us
I20260709 13:17:41.648344 12113 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.661412 11471 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.661841 11470 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 13 to 15, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.711364 11299 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 13 to 15, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.721097 11551 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:41.728874 12113 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:17:41.735077 17250 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:17:41.740357 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 57051us
I20260709 13:17:41.743747 11293 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.761289 17250 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Committing config change with OpId 13.13: config changed from index 12 to 13, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.764707 12113 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.13: config changed from index 12 to 13, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.769291 11471 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:17:41.776521 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:41.778218 17219 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:41.779827 17214 raft_consensus.cc:1064] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:41.782676 17214 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:41.798686 11471 raft_consensus.cc:2955] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.13: config changed from index 12 to 13, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.827227 11548 consensus_queue.cc:237] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:41.827589 11551 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:41.840174 12113 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:17:41.840441 11402 raft_consensus.cc:1275] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:17:41.842476 17223 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:17:41.851363 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:41.851114 17173 consensus_queue.cc:1048] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:17:41.853425 17190 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Committing config change with OpId 17.16: config changed from index 15 to 16, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.856891 12122 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Committing config change with OpId 17.16: config changed from index 15 to 16, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.854607 11402 raft_consensus.cc:2955] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Committing config change with OpId 17.16: config changed from index 15 to 16, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.865422 11475 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:41.867249 12113 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:41.885892 17215 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:17:41.886795 11296 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 12 to 13, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.918080 17249 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:41.959635 17224 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.961829 12122 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:41.962786 11475 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:41.968315 11394 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:41.974289 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:41.984611 11551 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:41.975725 11293 catalog_manager.cc:5697] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 15 to 16, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.003865 17131 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:17:42.025249 17229 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:42.036126 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b2eeda82ae3942d6bf0db873fa350bdc with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:42.045605 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:42.048863 12122 raft_consensus.cc:2955] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:42.069569 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62176.7us
I20260709 13:17:42.078001 17232 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:42.078652 11294 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.082309 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:42.167649 17299 tablet_replica.cc:333] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:17:42.174824 17299 raft_consensus.cc:2243] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:42.175720 17299 raft_consensus.cc:2272] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:42.178399 17297 tablet_replica.cc:333] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:42.197913 17297 raft_consensus.cc:2243] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:42.204219 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet b2eeda82ae3942d6bf0db873fa350bdc with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:42.182005 17299 ts_tablet_manager.cc:1916] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:42.217888 11393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:42.218540 11393 raft_consensus.cc:606] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:42.222064 17299 ts_tablet_manager.cc:1929] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.15
I20260709 13:17:42.222630 17299 log.cc:1199] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/b8eacd98a9dc4961a9fac3d2ae9dbd02
I20260709 13:17:42.223662 11545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:42.224275 11545 raft_consensus.cc:606] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:42.224416 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet b8eacd98a9dc4961a9fac3d2ae9dbd02 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:42.232196 11545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:42.232805 11545 raft_consensus.cc:606] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:42.247786 11548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:42.248399 11548 raft_consensus.cc:606] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:42.248808 11548 raft_consensus.cc:630] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Rejecting request to transfer leadership because tablet server 147201d22a694ca59f6e014f083a0892 is not a voter in the active config
I20260709 13:17:42.256340 17232 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.198931 17297 raft_consensus.cc:2272] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:42.284741 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:42.297845 11402 raft_consensus.cc:606] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:42.307328 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:42.296995 11548 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.309068 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:42.289471 17297 ts_tablet_manager.cc:1916] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:42.323997 17240 raft_consensus.cc:993] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:42.325718 17240 raft_consensus.cc:1081] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:42.328742 11401 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.335738 11402 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41730
I20260709 13:17:42.336467 11402 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.336890 11402 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.343398 17296 raft_consensus.cc:993] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:42.343982 17224 raft_consensus.cc:1081] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:42.352424 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:17:42.352999 11475 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.353322 11475 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:42.360234 11475 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.377146 11475 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:42.388747 17021 raft_consensus.cc:993] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:42.389281 17225 raft_consensus.cc:1081] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:42.392632 11477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:42.393253 11477 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.393607 11477 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.408807 11402 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.423780 11402 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:42.424623 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:42.425009 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:42.425253 11471 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.425765 11401 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:42.424557 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:42.427495 11548 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:42.427742 11548 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:42.431839 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:42.432402 11545 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:42.432665 11545 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:42.437412 11545 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.441916 11477 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.446655 11471 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:42.447964 11401 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:42.449003 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:42.451187 11298 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.452761 11477 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:42.457182 11545 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.459743 11548 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.469560 17298 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:42.469771 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:42.470223 17298 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:42.470355 11551 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.471204 17298 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.474030 12122 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.476609 17021 raft_consensus.cc:993] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:42.477108 17021 raft_consensus.cc:1081] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:42.478283 11548 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:42.487597 12113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:17:42.488216 12113 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.488561 12113 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.495328 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:42.495903 11401 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:42.498769 11401 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:17:42.501950 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51802.8us
I20260709 13:17:42.502287 11548 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:17:42.507419 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:42.515529 17225 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:42.517171 11551 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:17:42.507799 12113 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:42.519155 17225 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:42.523744 11401 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.537886 11401 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.520120 17225 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.557741 17297 ts_tablet_manager.cc:1929] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.12
I20260709 13:17:42.558290 17297 log.cc:1199] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/6a6ebf03c40d4f1fa404e0fe9f8f9fc5
I20260709 13:17:42.563325 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:42.574229 17298 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:42.573942 11294 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 12 to 13, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.575069 17298 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:42.575851 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:42.575963 17298 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:42.577116 11402 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:42.577497 11402 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:42.578336 11402 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:42.579504 11402 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.597031 11477 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:42.602692 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:42.603724 11401 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:17:42.598030 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:42.641115 11548 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:42.608453 12123 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:42.650223 11548 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:17:42.597554 12113 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:42.610680 11545 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:42.616721 11400 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:42.611677 17297 tablet_replica.cc:333] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:42.655304 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:42.656208 17252 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:42.656680 17252 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:42.657649 17252 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:42.627336 11294 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 8 to 9, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 9 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.651460 17281 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:42.689157 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 16 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.698248 17281 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:17:42.721815 17298 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.726953 11296 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 16, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 16 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.733609 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08ebf4312af74188830a07342d0dc119 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:42.746014 12113 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.749315 11400 raft_consensus.cc:2955] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:42.759865 17297 raft_consensus.cc:2243] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:42.760946 17297 raft_consensus.cc:2272] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:42.632072 11402 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:17:42.802743 11298 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 10 to 11, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 13 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.811586 17297 ts_tablet_manager.cc:1916] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:42.851830 17297 ts_tablet_manager.cc:1929] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.11
I20260709 13:17:42.852487 17297 log.cc:1199] T b2eeda82ae3942d6bf0db873fa350bdc P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/b2eeda82ae3942d6bf0db873fa350bdc
I20260709 13:17:42.854214 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet b2eeda82ae3942d6bf0db873fa350bdc (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:42.874872 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 08ebf4312af74188830a07342d0dc119 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:51266
I20260709 13:17:42.916200 17308 tablet_replica.cc:333] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:17:42.917344 17308 raft_consensus.cc:2243] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:42.918154 17308 raft_consensus.cc:2272] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:42.937072 17308 ts_tablet_manager.cc:1916] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:42.946550 11548 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:42.948586 17214 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:42.968168 11401 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:42.971407 17211 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:42.993136 11475 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:42.995011 17225 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:43.026057 17308 ts_tablet_manager.cc:1929] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.10
I20260709 13:17:43.027161 17308 log.cc:1199] T 08ebf4312af74188830a07342d0dc119 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/08ebf4312af74188830a07342d0dc119
I20260709 13:17:43.028842 11545 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:43.029069 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 08ebf4312af74188830a07342d0dc119 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:43.030499 17281 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:43.089064 11551 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:43.123721 11548 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:43.131703 17252 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:43.156247 11402 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:43.161144 17232 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:43.191164 11400 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:43.193954 17316 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:43.216270 17021 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:43.310514 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:44.220981 11400 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:17:44.249765 11366 consensus_peers.cc:597] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 930d965be7eb49448fa23456beb9925a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:44.249854 11551 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:44.251287 11475 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:44.251801 17225 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:44.253091 17021 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:44.357292 17225 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:44.387773 11549 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:44.430580 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:45.030117 11549 consensus_meta.cc:287] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Time spent flushing consensus metadata: real 0.628s	user 0.013s	sys 0.013s
I20260709 13:17:45.056125 11293 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:45.097623 11440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.017s
W20260709 13:17:45.098150 11440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.005s	sys 0.018s
W20260709 13:17:45.102059 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46762.1us
I20260709 13:17:45.160074 11475 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:17:45.211320 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49234.7us
I20260709 13:17:45.211709 11476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:45.212311 11476 raft_consensus.cc:606] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:45.231334 11471 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.239123 11548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:45.239794 11548 raft_consensus.cc:606] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:17:45.242914 11441 consensus_peers.cc:597] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2dfc6e65bda4ed2a39f4bce45ec85e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.244546 17349 ts_tablet_manager.cc:933] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:17:45.246238 11402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:45.246770 11402 raft_consensus.cc:606] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:45.248016 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:45.248183 11400 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:45.249727 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:45.251302 17331 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:45.272028 17342 raft_consensus.cc:993] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:45.272522 17342 raft_consensus.cc:1081] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:45.274255 11545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:45.274783 11545 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:45.275151 11545 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.304731 11550 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:45.316238 17211 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:45.295663 17349 tablet_copy_client.cc:323] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:17:45.328830 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 930d965be7eb49448fa23456beb9925a from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:17:45.329416 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 930d965be7eb49448fa23456beb9925a from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-930d965be7eb49448fa23456beb9925a
I20260709 13:17:45.332026 17331 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.338132 17350 raft_consensus.cc:993] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:45.338676 17350 raft_consensus.cc:1081] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:45.339429 11545 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.340662 11475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:17:45.341216 11475 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:45.341521 11475 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.353240 11412 tablet_copy_source_session.cc:215] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.354410 11475 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:45.357293 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:45.357884 11400 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.362802 11545 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:45.362833 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:45.363867 11402 raft_consensus.cc:3055] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:45.364209 11402 raft_consensus.cc:740] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:45.364989 11402 consensus_queue.cc:260] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.366273 11402 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.368654 11401 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.370159 11393 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:45.377195 11475 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:45.377956 11400 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:45.378933 17349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930d965be7eb49448fa23456beb9925a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.379405 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:45.387190 17211 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:45.389101 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:45.389719 12113 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:45.398576 12113 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:45.395454 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:45.405351 11545 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:45.405710 11545 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:45.406548 11545 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:45.408766 11545 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.409411 17211 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:45.413025 17211 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:45.419200 11402 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:17:45.420384 11516 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:45.421340 17352 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:45.421844 17352 raft_consensus.cc:697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:45.422693 17352 consensus_queue.cc:237] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.428004 11545 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:17:45.442401 17349 tablet_copy_client.cc:806] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.454550 11548 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:17:45.454545 11366 consensus_peers.cc:597] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38b39fd56a8d45f7b56a9ac114b66d2b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.455098 11475 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:45.455925 17225 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:45.457305 17342 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:45.449115 17349 tablet_copy_client.cc:670] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
W20260709 13:17:45.463358 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800365f20 after lost signal to thread 10255
W20260709 13:17:45.465978 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800365800 after lost signal to thread 11334
W20260709 13:17:45.467167 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08000b78e0 after lost signal to thread 11426
W20260709 13:17:45.468423 11498 debug-util.cc:398] Leaking SignalData structure 0x7b0800164440 after lost signal to thread 11501
W20260709 13:17:45.469748 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08003c9b20 after lost signal to thread 11575
W20260709 13:17:45.471151 11498 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6640 after lost signal to thread 12147
I20260709 13:17:45.470822 17349 tablet_copy_client.cc:538] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.479242 11547 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.500393 11475 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.505599 17331 raft_consensus.cc:993] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:45.506943 17211 raft_consensus.cc:1081] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:45.512372 11401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:45.512933 11401 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:45.513327 11401 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.495586 17342 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.521085 11401 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.524251 11401 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:45.525954 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:45.526558 12123 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.526795 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:45.527361 11476 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:45.527616 11476 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:45.528342 11476 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.529559 11476 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:45.538614 11476 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:45.502802 17349 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:45.549085 11365 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:45.549961 17225 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:45.550465 17225 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:45.551378 17225 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:45.567971 11548 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.575728 11296 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 9, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.585534 12123 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:17:45.615425 11298 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 15 to 16, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.627593 11547 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:45.640682 11293 catalog_manager.cc:5697] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 10 to 11, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 11 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.672339 11296 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.755556 17349 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:45.758659 17349 tablet_bootstrap.cc:492] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:45.759433 17349 ts_tablet_manager.cc:1403] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.257s	user 0.044s	sys 0.004s
I20260709 13:17:45.762429 11293 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 16, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 16 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:45.766973 11515 consensus_peers.cc:597] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe707c46816d4d84bf4ead0d04de2b26. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.770092 17349 raft_consensus.cc:359] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:45.776993 17349 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:45.777786 17349 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:45.788434 17349 ts_tablet_manager.cc:1434] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:17:45.791646 11393 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:45.795449 17352 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:45.813935 11471 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:45.820576 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-930d965be7eb49448fa23456beb9925a received from {username='slave'} at 127.0.0.1:50180
I20260709 13:17:45.821106 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-930d965be7eb49448fa23456beb9925a on tablet 930d965be7eb49448fa23456beb9925a with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:45.822073 17348 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:45.839879 17352 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.842605 11401 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.847661 11547 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:45.853508 17298 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:45.865952 17349 ts_tablet_manager.cc:933] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:45.860963 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.887254 17349 tablet_copy_client.cc:323] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:17:45.889199 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:17:45.889971 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-b2dfc6e65bda4ed2a39f4bce45ec85e6
I20260709 13:17:45.913859 11471 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:45.917632 11401 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:45.934862 11487 tablet_copy_source_session.cc:215] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.939138 17349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.969766 11401 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:45.972023 17337 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:45.944204 17354 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:17:46.001307 12123 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:46.009394 17377 ts_tablet_manager.cc:933] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:17:46.006912 17225 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:17:46.015862 12123 raft_consensus.cc:1275] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:46.022565 17352 consensus_queue.cc:1048] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:17:46.091306 17377 tablet_copy_client.cc:323] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:17:46.233301 11471 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:46.250643 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:46.260799 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 38b39fd56a8d45f7b56a9ac114b66d2b from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:17:46.261445 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 38b39fd56a8d45f7b56a9ac114b66d2b from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-38b39fd56a8d45f7b56a9ac114b66d2b
I20260709 13:17:46.261848 17235 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:46.269325 11412 tablet_copy_source_session.cc:215] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:46.322896 12118 raft_consensus.cc:1217] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.10->[15.11-15.11]   Dedup: 15.11->[]
W20260709 13:17:46.327260 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 67969.2us
W20260709 13:17:46.343384 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 125645us
W20260709 13:17:46.360656 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 164663us
W20260709 13:17:46.464900 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 125879us
W20260709 13:17:46.491118 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 150319us
I20260709 13:17:46.510331 17377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b39fd56a8d45f7b56a9ac114b66d2b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:46.539567 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 195926us
W20260709 13:17:46.559741 17394 log.cc:927] Time spent T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:17:46.574254 17349 tablet_copy_client.cc:806] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
W20260709 13:17:46.605572 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 269098us
W20260709 13:17:46.607110 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 58170.2us
I20260709 13:17:46.574929 17349 tablet_copy_client.cc:670] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
W20260709 13:17:46.642351 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 70123.4us
W20260709 13:17:46.783433 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 140061us
I20260709 13:17:46.844348 17415 ts_tablet_manager.cc:933] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:46.892302 17415 tablet_copy_client.cc:323] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:46.900826 17377 tablet_copy_client.cc:806] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:46.901497 17377 tablet_copy_client.cc:670] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
W20260709 13:17:46.903973 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 65331.7us
I20260709 13:17:46.904163 17349 tablet_copy_client.cc:538] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:17:46.941140 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 94500.2us
I20260709 13:17:46.948935 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet fe707c46816d4d84bf4ead0d04de2b26 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:46.949534 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet fe707c46816d4d84bf4ead0d04de2b26 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-fe707c46816d4d84bf4ead0d04de2b26
I20260709 13:17:47.006768 17349 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:47.007609 17377 tablet_copy_client.cc:538] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:47.028604 11561 tablet_copy_source_session.cc:215] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:47.034192 17415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe707c46816d4d84bf4ead0d04de2b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:47.038801 17377 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:47.053125 17330 raft_consensus.cc:1064] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:47.056289 17330 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:47.080274 12118 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:47.081610 11549 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:47.081683 17420 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:47.084930 11471 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:47.085428 17420 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:47.090723 17420 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:47.131469 17415 tablet_copy_client.cc:806] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:47.132125 17415 tablet_copy_client.cc:670] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:47.152179 17415 tablet_copy_client.cc:538] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:47.167958 17330 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:47.176048 12119 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:47.192798 11293 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:47.193080 11470 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:47.236959 17415 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:47.304054 11549 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:47.365108 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:47.399389 17349 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:17:47.407256 17349 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:47.408776 17349 tablet_bootstrap.cc:492] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
W20260709 13:17:47.405705 17377 log.cc:927] Time spent T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:17:47.415185 17349 ts_tablet_manager.cc:1403] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.409s	user 0.038s	sys 0.010s
I20260709 13:17:47.595151 17349 raft_consensus.cc:359] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:47.596266 17349 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:47.596988 17349 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:47.600749 17349 ts_tablet_manager.cc:1434] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.183s	user 0.005s	sys 0.003s
I20260709 13:17:47.620028 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b2dfc6e65bda4ed2a39f4bce45ec85e6 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:17:47.620820 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-b2dfc6e65bda4ed2a39f4bce45ec85e6 on tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 with peer b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:17:47.649115 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 100344us
I20260709 13:17:47.812042 17377 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:47.813241 17377 tablet_bootstrap.cc:492] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:47.814570 17377 ts_tablet_manager.cc:1403] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.776s	user 0.045s	sys 0.009s
W20260709 13:17:47.846745 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 111462us
I20260709 13:17:47.862071 12119 raft_consensus.cc:1217] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 13:17:47.947753 17377 raft_consensus.cc:359] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:47.948923 17377 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:47.949625 17377 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:47.972544 11401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:47.973284 11401 raft_consensus.cc:606] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:47.981827 11477 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:47.982421 11477 raft_consensus.cc:606] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:47.987633 11550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:47.988353 11550 raft_consensus.cc:606] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:47.990001 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:47.991647 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:47.998229 12119 raft_consensus.cc:1217] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:17:47.995187 17377 ts_tablet_manager.cc:1434] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.180s	user 0.012s	sys 0.000s
I20260709 13:17:48.024487 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-38b39fd56a8d45f7b56a9ac114b66d2b received from {username='slave'} at 127.0.0.1:50180
I20260709 13:17:48.025141 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-38b39fd56a8d45f7b56a9ac114b66d2b on tablet 38b39fd56a8d45f7b56a9ac114b66d2b with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:48.047576 17330 raft_consensus.cc:993] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:48.048081 17330 raft_consensus.cc:1081] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:48.050021 11476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:48.050614 11476 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:48.051038 11476 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.065460 17415 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:48.066571 17415 tablet_bootstrap.cc:492] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:48.067499 17415 ts_tablet_manager.cc:1403] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.831s	user 0.059s	sys 0.033s
I20260709 13:17:48.069628 11476 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.070629 17415 raft_consensus.cc:359] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:48.079320 17415 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:48.080698 11476 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:48.080291 17415 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:48.081578 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:48.086102 11401 raft_consensus.cc:3055] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:48.086434 11401 raft_consensus.cc:740] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:48.084093 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:48.087817 17415 ts_tablet_manager.cc:1434] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:17:48.087316 11401 consensus_queue.cc:260] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.089092 11401 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.087828 11548 raft_consensus.cc:3060] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:48.097810 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-fe707c46816d4d84bf4ead0d04de2b26 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:48.098366 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-fe707c46816d4d84bf4ead0d04de2b26 on tablet fe707c46816d4d84bf4ead0d04de2b26 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:48.107540 11401 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:48.111101 11442 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:48.112967 17332 raft_consensus.cc:2804] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:48.113579 17332 raft_consensus.cc:697] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:48.114320 11548 raft_consensus.cc:2468] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:17:48.114498 17332 consensus_queue.cc:237] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.122726 17211 raft_consensus.cc:993] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:48.124454 17211 raft_consensus.cc:1081] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:48.131387 11394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:48.132169 11394 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:48.132614 11394 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.183871 11393 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:48.184552 12119 raft_consensus.cc:1217] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 13:17:48.204823 11293 catalog_manager.cc:5697] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.205683 11394 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.213377 11548 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:48.215148 17420 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:48.210321 12119 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:48.231832 17021 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:48.241858 17021 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.248386 11550 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.262169 11394 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:48.262996 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:48.263628 11477 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:48.263932 11477 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:48.267863 11477 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.269292 11477 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.280215 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:48.280882 11545 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.304774 17418 raft_consensus.cc:1064] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:48.312959 17383 raft_consensus.cc:993] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:48.313637 17383 raft_consensus.cc:1081] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:48.320377 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 930d965be7eb49448fa23456beb9925a with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:48.330538 11477 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:17:48.330476 11294 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 12 to 13, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.332019 11365 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:48.332247 11394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41730
I20260709 13:17:48.332847 11394 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:48.333174 11394 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:48.333878 17420 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:48.334414 17420 raft_consensus.cc:697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:48.334867 17418 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:48.335350 17420 consensus_queue.cc:237] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.340384 11394 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.340261 12118 raft_consensus.cc:2955] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.346911 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 930d965be7eb49448fa23456beb9925a with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:48.356766 11394 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:48.359826 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:48.360459 11550 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:48.360760 11550 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:48.361747 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:48.362308 11477 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:48.365659 17442 tablet_replica.cc:333] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:17:48.368956 11550 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.376592 11550 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:48.384899 11545 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:17:48.391408 11550 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:17:48.396811 17442 raft_consensus.cc:2243] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.398095 17442 raft_consensus.cc:2272] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.406026 11294 catalog_manager.cc:5697] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 10, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.413440 11367 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:48.425107 17420 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:48.425735 17442 ts_tablet_manager.cc:1916] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:48.426043 17420 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:48.437335 17330 raft_consensus.cc:1064] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:48.447059 17330 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:48.449688 17420 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:48.465262 11477 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
W20260709 13:17:48.572916 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 44566.2us
I20260709 13:17:48.577126 11294 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 14 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 15 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.579788 12119 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:48.584092 17021 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:48.593456 11550 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:48.597030 12113 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:48.602577 11550 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:48.637910 17447 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:48.644645 17442 ts_tablet_manager.cc:1929] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.12
I20260709 13:17:48.645294 17442 log.cc:1199] T 930d965be7eb49448fa23456beb9925a P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/930d965be7eb49448fa23456beb9925a
I20260709 13:17:48.653964 17434 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:17:48.647400 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 930d965be7eb49448fa23456beb9925a (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:48.664423 11393 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:48.668902 17337 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:48.687880 11393 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:48.693274 17447 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:48.696939 11549 raft_consensus.cc:1275] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:48.698603 17446 consensus_queue.cc:1048] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:48.687467 11477 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:17:48.761145 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46240.7us
W20260709 13:17:48.793807 17453 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:17:48.854399 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 66624.1us
I20260709 13:17:48.855896 11477 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:48.868593 17211 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:48.888087 17021 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:48.917709 17418 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.927803 17330 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:48.928288 17436 raft_consensus.cc:1064] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:48.964613 17404 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.979257 11549 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.980605 11548 raft_consensus.cc:1275] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:48.988181 11396 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:48.997159 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:48.995301 12119 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.019265 17455 consensus_queue.cc:1048] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:17:49.032168 17436 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:49.061228 11294 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.052968 11293 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.077755 11471 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:49.095913 11549 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:49.099825 11477 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.112965 17225 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:17:49.125202 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 181281us
W20260709 13:17:49.126762 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 192375us
I20260709 13:17:49.138442 11549 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.159085 17420 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:49.170580 11397 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:49.181581 17436 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:49.210400 11476 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:49.232872 12119 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:17:49.241330 17448 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:49.259536 12123 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.324209 17383 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:17:49.353425 17436 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.380432 11476 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.384539 11477 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:49.392161 12119 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:49.394686 17461 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:49.402562 11542 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:49.419597 17468 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.435763 12113 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.457993 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.474261 11396 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:49.475183 17467 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:17:49.480921 12123 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.484663 12118 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:49.486112 11477 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:49.488507 17330 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:49.491358 17330 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:49.491791 11547 raft_consensus.cc:2955] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.509857 11395 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.516379 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.521101 17434 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.525141 11298 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 10 to 11, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 12 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.534536 11476 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.562212 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:49.599889 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38b39fd56a8d45f7b56a9ac114b66d2b with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.606429 12118 raft_consensus.cc:2955] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.619606 17487 tablet_replica.cc:333] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:49.620764 17487 raft_consensus.cc:2243] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.621626 17487 raft_consensus.cc:2272] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.619866 11296 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.639979 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 38b39fd56a8d45f7b56a9ac114b66d2b with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:51266
I20260709 13:17:49.648252 17487 ts_tablet_manager.cc:1916] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.657542 17488 tablet_replica.cc:333] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:17:49.659082 17488 raft_consensus.cc:2243] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.660099 17488 raft_consensus.cc:2272] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.739966 17487 ts_tablet_manager.cc:1929] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 13:17:49.740581 17487 log.cc:1199] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/b2dfc6e65bda4ed2a39f4bce45ec85e6
I20260709 13:17:49.741066 17488 ts_tablet_manager.cc:1916] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.742563 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet b2dfc6e65bda4ed2a39f4bce45ec85e6 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:49.836714 17488 ts_tablet_manager.cc:1929] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 13:17:49.837358 17488 log.cc:1199] T 38b39fd56a8d45f7b56a9ac114b66d2b P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/38b39fd56a8d45f7b56a9ac114b66d2b
I20260709 13:17:49.848526 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 38b39fd56a8d45f7b56a9ac114b66d2b (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:49.929222 11547 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:49.939934 12123 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:49.943162 17392 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:49.956178 11477 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:49.957639 17448 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:49.956951 17449 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.962393 11477 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.972025 12118 raft_consensus.cc:2955] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:49.975778 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fe707c46816d4d84bf4ead0d04de2b26 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.988531 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 11 to 12, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.026882 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet fe707c46816d4d84bf4ead0d04de2b26 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:50.032760 17487 tablet_replica.cc:333] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:50.033898 17487 raft_consensus.cc:2243] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:50.034794 17487 raft_consensus.cc:2272] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:50.039948 17487 ts_tablet_manager.cc:1916] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:50.059713 17487 ts_tablet_manager.cc:1929] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.11
I20260709 13:17:50.060140 17487 log.cc:1199] T fe707c46816d4d84bf4ead0d04de2b26 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/fe707c46816d4d84bf4ead0d04de2b26
I20260709 13:17:50.061817 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet fe707c46816d4d84bf4ead0d04de2b26 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:50.374909 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:50.476209 11395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f946b7973fe482681135635493ebd48"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:50.476822 11395 raft_consensus.cc:606] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:50.487015 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:50.487669 11396 raft_consensus.cc:606] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:50.494156 11470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:50.494768 11470 raft_consensus.cc:606] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:50.497576 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:50.499321 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:50.499464 17434 raft_consensus.cc:993] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:50.499989 17434 raft_consensus.cc:1081] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:17:50.502679 11547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43430
I20260709 13:17:50.503979 11547 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.504364 11547 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:50.511569 17467 raft_consensus.cc:993] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:50.512286 17461 raft_consensus.cc:1081] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:50.514111 12118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55074
I20260709 13:17:50.514621 12118 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.514957 12118 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.521554 12118 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.523593 11547 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.527808 12118 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:50.529611 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:50.541179 11396 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.543176 11547 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:50.548352 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:50.548415 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:50.563078 11395 raft_consensus.cc:3055] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:50.550027 11396 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:17:50.563210 12118 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:50.534595 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:50.563475 11395 raft_consensus.cc:740] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:50.563958 11476 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:50.564308 11476 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:50.566140 11395 consensus_queue.cc:260] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.568536 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:50.566063 11476 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.570164 11476 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.574268 17347 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:50.574828 17347 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:50.575740 17347 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.583705 12118 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:17:50.587146 11476 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:17:50.587316 11395 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:50.597828 11515 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:50.599673 17449 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:50.600227 17449 raft_consensus.cc:697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:50.601142 17449 consensus_queue.cc:237] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:50.604666 11395 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:17:50.609966 11293 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 17 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.626919 11298 catalog_manager.cc:5697] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 17 to 18, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.796682 17434 raft_consensus.cc:993] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:50.797178 17434 raft_consensus.cc:1081] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:50.798849 11476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f946b7973fe482681135635493ebd48"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:50.799402 11476 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:50.799722 11476 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.812703 11476 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:50.815608 11476 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:50.818084 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:50.818652 11547 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.824381 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:50.825044 11395 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:50.825338 11395 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:50.826074 11395 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:50.827332 11395 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:50.834465 11547 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:50.842139 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:17:50.843065 17497 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:50.843549 17497 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:50.844365 17497 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:50.854648 11395 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:50.874389 11298 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 16 to 17, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.105501 11470 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:51.108860 17489 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:51.114640 11396 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:51.120152 17500 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:51.180965 11396 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:51.183020 17489 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:51.189563 12118 raft_consensus.cc:1275] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:51.198640 17500 consensus_queue.cc:1048] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:51.370622 11547 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:51.390161 17461 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:17:51.399962 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 114999us
I20260709 13:17:51.439123 11396 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:51.446636 17497 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:17:51.470178 11547 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:51.500312 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:51.561648 11515 consensus_peers.cc:597] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eeb668f66ce344e9acdf8834c2b1d3a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.602530 11396 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:51.610507 17448 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:51.614956 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 117329us
I20260709 13:17:51.652436 11476 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:17:51.662184 17498 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:51.665688 17510 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.683604 11396 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.715365 11470 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.710314 11293 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.721328 11395 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:51.766826 11547 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:51.769912 17455 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:51.780805 11470 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:51.782208 17225 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:17:51.815806 11366 consensus_peers.cc:597] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7abd40007d144575bd9e785e609a62df. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.835104 17527 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.856973 11547 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.876868 11476 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:51.885272 11470 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:51.924823 11296 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 10, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:51.934687 11441 consensus_peers.cc:597] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b37b6683af04823b0814ba56c4ea803. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.964581 11547 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:51.967365 11395 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:51.973780 17497 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:51.978916 17497 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:52.000988 17535 ts_tablet_manager.cc:933] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:52.024969 17535 tablet_copy_client.cc:323] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:52.026909 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet eeb668f66ce344e9acdf8834c2b1d3a3 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:52.027526 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet eeb668f66ce344e9acdf8834c2b1d3a3 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-eeb668f66ce344e9acdf8834c2b1d3a3
I20260709 13:17:52.036362 11561 tablet_copy_source_session.cc:215] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.038632 17521 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.040294 17535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb668f66ce344e9acdf8834c2b1d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.049355 11396 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.085691 17535 tablet_copy_client.cc:806] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.086462 17535 tablet_copy_client.cc:670] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.095741 11542 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.107707 17535 tablet_copy_client.cc:538] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.110412 11293 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.118865 17535 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:52.122162 11396 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:17:52.181171 11366 consensus_peers.cc:597] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 74bea11400ce497283dceb7d21b6e728. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.187734 11476 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:52.192034 11545 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:17:52.197613 17434 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:52.199555 17434 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:52.215108 17434 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.217635 11470 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.245426 11296 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.251715 17535 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.252815 17535 tablet_bootstrap.cc:492] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:52.253587 17535 ts_tablet_manager.cc:1403] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.135s	user 0.032s	sys 0.022s
I20260709 13:17:52.256750 17535 raft_consensus.cc:359] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:52.257663 17535 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:52.258324 17535 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:52.261628 17535 ts_tablet_manager.cc:1434] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:52.266851 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-eeb668f66ce344e9acdf8834c2b1d3a3 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:52.267328 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-eeb668f66ce344e9acdf8834c2b1d3a3 on tablet eeb668f66ce344e9acdf8834c2b1d3a3 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:52.301779 11547 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:52.371600 17535 ts_tablet_manager.cc:933] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:17:52.391001 17535 tablet_copy_client.cc:323] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:17:52.392602 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 7abd40007d144575bd9e785e609a62df from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:17:52.393129 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 7abd40007d144575bd9e785e609a62df from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-7abd40007d144575bd9e785e609a62df
I20260709 13:17:52.404184 11412 tablet_copy_source_session.cc:215] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.409890 17535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abd40007d144575bd9e785e609a62df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.441846 17535 tablet_copy_client.cc:806] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.442737 17535 tablet_copy_client.cc:670] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.466899 17535 tablet_copy_client.cc:538] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.489617 17535 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:52.540396 17544 ts_tablet_manager.cc:933] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:17:52.551990 12119 raft_consensus.cc:1217] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 13:17:52.560472 17544 tablet_copy_client.cc:323] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.2:36161
I20260709 13:17:52.562619 11487 tablet_copy_service.cc:140] P 34fe1737ebb840fb9746980f649cc0a6: Received BeginTabletCopySession request for tablet 9b37b6683af04823b0814ba56c4ea803 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:43866)
I20260709 13:17:52.563253 11487 tablet_copy_service.cc:161] P 34fe1737ebb840fb9746980f649cc0a6: Beginning new tablet copy session on tablet 9b37b6683af04823b0814ba56c4ea803 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:43866: session id = b3abf4b5ee724203ad1226f5c46bfe46-9b37b6683af04823b0814ba56c4ea803
I20260709 13:17:52.577479 11487 tablet_copy_source_session.cc:215] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.633724 17544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b37b6683af04823b0814ba56c4ea803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.651800 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:52.679286 17535 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.680245 17535 tablet_bootstrap.cc:492] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:52.680886 17535 ts_tablet_manager.cc:1403] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.192s	user 0.036s	sys 0.044s
I20260709 13:17:52.702167 17535 raft_consensus.cc:359] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:52.703364 17535 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:52.705897 17535 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:52.708097 17535 ts_tablet_manager.cc:1434] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:17:52.711926 17547 ts_tablet_manager.cc:933] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:17:52.728117 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-7abd40007d144575bd9e785e609a62df received from {username='slave'} at 127.0.0.1:50180
I20260709 13:17:52.728595 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-7abd40007d144575bd9e785e609a62df on tablet 7abd40007d144575bd9e785e609a62df with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:52.786898 17544 tablet_copy_client.cc:806] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.801337 17544 tablet_copy_client.cc:670] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.802134 17547 tablet_copy_client.cc:323] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:17:52.811060 17544 tablet_copy_client.cc:538] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.819387 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 74bea11400ce497283dceb7d21b6e728 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:17:52.819973 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 74bea11400ce497283dceb7d21b6e728 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-74bea11400ce497283dceb7d21b6e728
I20260709 13:17:52.823554 17544 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:52.827860 11412 tablet_copy_source_session.cc:215] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.831241 17547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bea11400ce497283dceb7d21b6e728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.850239 17547 tablet_copy_client.cc:806] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.850948 17547 tablet_copy_client.cc:670] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.858662 17547 tablet_copy_client.cc:538] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.871526 11547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:17:52.872174 11547 raft_consensus.cc:606] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:52.875367 11470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:52.875957 11470 raft_consensus.cc:606] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:52.888152 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:52.890329 17547 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:52.895455 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:52.989647 17544 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.990662 17544 tablet_bootstrap.cc:492] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:52.994776 17544 ts_tablet_manager.cc:1403] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.171s	user 0.036s	sys 0.012s
I20260709 13:17:52.997856 17544 raft_consensus.cc:359] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:53.003257 17544 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:53.003669 12118 raft_consensus.cc:1217] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:17:53.004050 17544 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:53.009320 17544 ts_tablet_manager.cc:1434] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:17:53.018478 12119 raft_consensus.cc:1217] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.11->[16.12-16.12]   Dedup: 16.12->[]
I20260709 13:17:53.023753 11487 tablet_copy_service.cc:342] P 34fe1737ebb840fb9746980f649cc0a6: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-9b37b6683af04823b0814ba56c4ea803 received from {username='slave'} at 127.0.0.1:43866
I20260709 13:17:53.024233 11487 tablet_copy_service.cc:434] P 34fe1737ebb840fb9746980f649cc0a6: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-9b37b6683af04823b0814ba56c4ea803 on tablet 9b37b6683af04823b0814ba56c4ea803 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:53.038391 17497 raft_consensus.cc:993] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:53.038925 17497 raft_consensus.cc:1081] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:53.047245 11396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:53.048022 11396 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.048528 11396 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.088176 11396 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:53.093133 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:53.093762 11470 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:53.094048 11470 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:53.094887 11470 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:53.096098 11470 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.105794 17547 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:53.107635 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:53.117419 11545 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.124854 11545 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:53.108080 11396 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:53.126137 17547 tablet_bootstrap.cc:492] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:53.128402 17547 ts_tablet_manager.cc:1403] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.258s	user 0.031s	sys 0.012s
I20260709 13:17:53.126147 11470 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:17:53.135553 11365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:53.136399 17531 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:53.136974 17531 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:53.131656 17547 raft_consensus.cc:359] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:53.137849 17531 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:53.140341 17498 raft_consensus.cc:1064] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:53.138331 17547 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:53.142504 17547 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:53.142935 17498 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.168547 17547 ts_tablet_manager.cc:1434] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.040s	user 0.003s	sys 0.004s
I20260709 13:17:53.176415 12119 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:53.178273 11396 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:53.178534 17510 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:53.184716 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-74bea11400ce497283dceb7d21b6e728 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:17:53.185161 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-74bea11400ce497283dceb7d21b6e728 on tablet 74bea11400ce497283dceb7d21b6e728 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:53.187752 17449 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:53.193301 11470 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:53.208563 17510 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
W20260709 13:17:53.392654 17555 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:17:53.407612 11298 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:53.423473 17552 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:17:53.432226 17556 log.cc:927] Time spent T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:53.449271 17498 raft_consensus.cc:993] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:53.449898 17498 raft_consensus.cc:1081] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:53.453701 11476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:17:53.454488 11476 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.454947 11476 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.465938 11476 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.477157 11476 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:17:53.489511 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 101771us
I20260709 13:17:53.526760 17562 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:53.540952 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 159462us
I20260709 13:17:53.544955 11470 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:53.553761 12118 raft_consensus.cc:1217] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.7->[13.8-13.8]   Dedup: 13.8->[]
I20260709 13:17:53.564435 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:17:53.565239 12123 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.587908 12123 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:53.606823 11471 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:53.622839 11441 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:53.625614 17531 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:53.623703 17570 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:53.629951 17570 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:53.639514 17225 raft_consensus.cc:1064] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:53.645977 17570 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.647991 11545 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:53.656049 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:53.655196 11396 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:53.656663 11542 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:53.656957 11542 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:17:53.657660 11542 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.658701 11542 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:53.663401 17559 raft_consensus.cc:1064] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:53.642653 17225 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.665848 17559 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:53.688416 12113 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:53.695200 12113 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:53.726740 11542 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:17:53.743867 11293 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:53.753782 12123 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEARNER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:17:53.753661 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 103792us
I20260709 13:17:53.756845 17558 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:53.763132 17434 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:53.766909 11545 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:53.767715 11396 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:53.769896 17570 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:53.770390 12123 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:53.779626 11542 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:53.772053 17527 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:17:53.782063 11476 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:53.786631 17531 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:53.788084 17559 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:53.899587 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:54.725580 17587 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:17:54.726344 17587 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:54.728883 17587 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:54.729548 17587 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:54.796206 11298 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 363) took 1432 ms. Trace:
I20260709 13:17:54.805222 11293 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 334) took 1152 ms. Trace:
I20260709 13:17:54.805877 11393 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:54.796769 11298 rpcz_store.cc:276] 0709 13:17:53.363394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.363562 (+   168us) service_pool.cc:224] Handling call
0709 13:17:54.796101 (+1432539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:53.622390 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:53.622919 (+   529us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:53.622951 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:17:53.622968 (+    17us) tablet.cc:662] Decoding operations
0709 13:17:53.623308 (+   340us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:53.623376 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:53.623407 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:53.623591 (+   184us) tablet.cc:701] Row locks acquired
0709 13:17:53.623609 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:17:53.623741 (+   132us) write_op.cc:270] Start()
0709 13:17:53.623848 (+   107us) write_op.cc:276] Timestamp: P: 1783603073623693 usec, L: 0
0709 13:17:53.623870 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:17:53.624498 (+   628us) log.cc:844] Serialized 532 byte log entry
0709 13:17:53.961880 (+337382us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.983831 (+ 21951us) write_op.cc:301] APPLY: starting
0709 13:17:53.983913 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:53.985016 (+  1103us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:53.985801 (+   785us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:53.988134 (+  2333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:53.989373 (+  1239us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:17:53.989425 (+    52us) write_op.cc:312] APPLY: finished
0709 13:17:53.989651 (+   226us) log.cc:844] Serialized 25 byte log entry
0709 13:17:53.991758 (+  2107us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.992660 (+   902us) write_op.cc:454] Released schema lock
0709 13:17:53.992709 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:53.993670 (+   961us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.993696 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":21303,"apply.run_cpu_time_us":3526,"apply.run_wall_time_us":9637,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":209719,"prepare.run_cpu_time_us":19406,"prepare.run_wall_time_us":160376,"raft.queue_time_us":216803,"raft.run_cpu_time_us":2528,"raft.run_wall_time_us":39424,"replication_time_us":324452,"thread_start_us":521613,"threads_started":4,"wal-append.queue_time_us":111385}]]}
I20260709 13:17:54.805632 11293 rpcz_store.cc:276] 0709 13:17:53.652534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.652685 (+   151us) service_pool.cc:224] Handling call
0709 13:17:53.737673 (+ 84988us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:17:54.805157 (+1067484us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:53.783044 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:53.783658 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:53.783692 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:17:53.783711 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:53.784086 (+   375us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:53.784134 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:53.784168 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:53.784377 (+   209us) tablet.cc:701] Row locks acquired
0709 13:17:53.784403 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:17:53.796777 (+ 12374us) write_op.cc:270] Start()
0709 13:17:53.796902 (+   125us) write_op.cc:276] Timestamp: P: 1783603073784513 usec, L: 0
0709 13:17:53.796925 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:17:53.800743 (+  3818us) log.cc:844] Serialized 587 byte log entry
0709 13:17:53.984332 (+183589us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.985354 (+  1022us) write_op.cc:301] APPLY: starting
0709 13:17:53.985416 (+    62us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:53.986503 (+  1087us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:53.986646 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:53.989066 (+  2420us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:53.990041 (+   975us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:17:53.990097 (+    56us) write_op.cc:312] APPLY: finished
0709 13:17:53.990271 (+   174us) log.cc:844] Serialized 25 byte log entry
0709 13:17:53.990449 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.990524 (+    75us) write_op.cc:454] Released schema lock
0709 13:17:53.990551 (+    27us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:53.990966 (+   415us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.991021 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58836,"reported_tablets":1,"spinlock_wait_cycles":190670720,"child_traces":[["op",{"apply.queue_time_us":837,"apply.run_cpu_time_us":3888,"apply.run_wall_time_us":5565,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":34855,"prepare.run_cpu_time_us":2734,"prepare.run_wall_time_us":18024,"replication_time_us":187280,"thread_start_us":666,"threads_started":1}]]}
I20260709 13:17:54.829892 17568 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:54.830454 17568 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:54.832653 17568 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:54.837249 17568 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:54.837785 17568 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:54.839898 17568 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:54.804039 17598 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:54.860853 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:54.871713 11551 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.001s	sys 0.010s
W20260709 13:17:54.873380 11551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.002s	sys 0.010s
I20260709 13:17:54.874298 11551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43430 (request call id 4310) took 1109 ms. Trace:
I20260709 13:17:54.880417 11294 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 351) took 1120 ms. Trace:
I20260709 13:17:54.885872 11294 rpcz_store.cc:276] 0709 13:17:53.759554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.760312 (+   758us) service_pool.cc:224] Handling call
0709 13:17:54.880361 (+1120049us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:17:54.874616 11551 rpcz_store.cc:276] 0709 13:17:53.765097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.765532 (+   435us) service_pool.cc:224] Handling call
0709 13:17:53.765831 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.766192 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.768153 (+  1961us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.768644 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.888356 (+119712us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:17:54.873768 (+985412us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.873991 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.874060 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.874100 (+    40us) raft_consensus.cc:1685] finished
0709 13:17:54.874116 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.874235 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89744,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":15838,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":233,"thread_start_us":1068916,"threads_started":2,"wal-append.queue_time_us":119950}
I20260709 13:17:54.923763 11543 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:54.929129 11296 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 227) took 1367 ms. Trace:
I20260709 13:17:54.929814 11296 rpcz_store.cc:276] 0709 13:17:53.562128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.562282 (+   154us) service_pool.cc:224] Handling call
0709 13:17:54.929066 (+1366784us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":496512}
I20260709 13:17:54.955907 17599 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:54.956702 17599 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:54.958153 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:54.959844 17599 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:54.881208 17600 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:54.970309 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:54.971149 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:54.969882 17600 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:54.977507 11545 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:54.979575 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:54.980171 17600 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:54.981076 11546 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:54.982879 11423 debug-util.cc:398] Leaking SignalData structure 0x7b0800335940 after lost signal to thread 10255
W20260709 13:17:54.984098 11423 debug-util.cc:398] Leaking SignalData structure 0x7b080034ea60 after lost signal to thread 11334
W20260709 13:17:54.985383 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5a80 after lost signal to thread 11426
W20260709 13:17:54.985666 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5a60 after lost signal to thread 11427
W20260709 13:17:54.986729 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08001b92c0 after lost signal to thread 11501
W20260709 13:17:54.988018 11423 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9860 after lost signal to thread 11575
W20260709 13:17:54.989286 11423 debug-util.cc:398] Leaking SignalData structure 0x7b080030d280 after lost signal to thread 12147
I20260709 13:17:54.989872 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:17:55.007876 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 45520.9us
I20260709 13:17:55.017081 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:55.020816 11395 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:55.030663 17564 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:55.045534 17564 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:55.046114 17564 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.050318 17564 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:55.053995 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:54.960024 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:55.068763 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.069723 17599 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:55.074249 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:55.091332 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:55.094131 17595 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:55.112187 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.114460 17606 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:55.128484 17580 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.149140 11296 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.163882 17599 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:17:55.164496 17599 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.167902 17599 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:55.168569 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 95391.2us
I20260709 13:17:55.169517 12085 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.173053 12113 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:55.174890 12123 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:55.201838 17599 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:55.209160 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:55.211776 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:55.208402 17599 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:17:55.213138 17599 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.215266 17599 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:55.216804 11471 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.223924 11546 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.232647 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:55.250084 17595 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.241403 11294 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.273658 11402 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:17:55.290642 11517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.305589 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:55.317510 12085 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.331648 17599 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:55.323845 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:55.334246 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:55.336145 17527 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:55.333876 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:17:55.338315 17527 raft_consensus.cc:1064] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:17:55.349071 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:17:55.350557 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.354413 17598 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:55.357142 17567 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:55.355650 17599 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:55.363183 17527 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 6, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.366684 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:55.372080 17598 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:55.381246 12122 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.386093 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eeb668f66ce344e9acdf8834c2b1d3a3 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:17:55.403371 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 52888.2us
I20260709 13:17:55.407984 11293 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 10 to 11, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.409988 12123 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.415513 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:17:55.417800 17568 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:55.426096 11476 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.428531 11546 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:55.429945 11471 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:55.437469 12119 raft_consensus.cc:2955] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.440925 12112 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:55.436195 17527 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:17:55.456327 17587 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:55.517623 11545 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:55.539888 17225 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:55.545459 12115 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:55.555982 17580 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.560835 11546 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.545534 17602 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:17:55.575526 17602 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.578078 17602 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:17:55.535601 17570 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:17:55.583967 11298 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 11 to 12, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 15 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.586812 17598 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:55.596474 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:17:55.601332 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:17:55.610147 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:55.611004 17598 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:55.612365 17624 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:17:55.619743 12115 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.623257 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet eeb668f66ce344e9acdf8834c2b1d3a3 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:59884
I20260709 13:17:55.623608 11396 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.637568 17606 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.644486 11477 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:55.647464 11477 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.652874 12115 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:55.653574 17225 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:55.656689 17527 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:17:55.672115 17527 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.680004 11543 raft_consensus.cc:2955] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.683763 12122 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.706498 11294 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.711300 11296 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.715178 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7abd40007d144575bd9e785e609a62df with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:55.701414 12112 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.723419 17630 tablet_replica.cc:333] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:17:55.725066 17630 raft_consensus.cc:2243] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.726037 17630 raft_consensus.cc:2272] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.734773 11284 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9b37b6683af04823b0814ba56c4ea803 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:55.749791 17630 ts_tablet_manager.cc:1916] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:55.766748 11298 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.768565 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b37b6683af04823b0814ba56c4ea803 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:55.778136 11477 raft_consensus.cc:2955] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.866487 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 7abd40007d144575bd9e785e609a62df with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:51266
I20260709 13:17:55.866364 11396 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:55.882755 12112 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:55.887979 17434 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:55.888247 11476 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:55.891379 17631 tablet_replica.cc:333] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:55.892256 17580 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:55.917701 17631 raft_consensus.cc:2243] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.918650 17631 raft_consensus.cc:2272] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.920817 17630 ts_tablet_manager.cc:1929] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:17:55.921296 17630 log.cc:1199] T eeb668f66ce344e9acdf8834c2b1d3a3 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/eeb668f66ce344e9acdf8834c2b1d3a3
I20260709 13:17:55.922164 17631 ts_tablet_manager.cc:1916] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:55.923035 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet eeb668f66ce344e9acdf8834c2b1d3a3 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:55.931857 17632 tablet_replica.cc:333] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:17:55.932941 17632 raft_consensus.cc:2243] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.932052 17603 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.945477 12115 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.958292 17632 raft_consensus.cc:2272] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.968425 17632 ts_tablet_manager.cc:1916] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:55.981894 11296 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 13 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:55.987536 11476 raft_consensus.cc:2955] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:56.002331 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 74bea11400ce497283dceb7d21b6e728 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51266
I20260709 13:17:56.023638 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 74bea11400ce497283dceb7d21b6e728 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:56.032956 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:56.035817 17632 ts_tablet_manager.cc:1929] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 13:17:56.036300 17632 log.cc:1199] T 7abd40007d144575bd9e785e609a62df P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/7abd40007d144575bd9e785e609a62df
I20260709 13:17:56.038046 17632 tablet_replica.cc:333] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:17:56.037974 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 7abd40007d144575bd9e785e609a62df (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:56.044823 17631 ts_tablet_manager.cc:1929] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.13
I20260709 13:17:56.064806 17631 log.cc:1199] T 9b37b6683af04823b0814ba56c4ea803 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/9b37b6683af04823b0814ba56c4ea803
I20260709 13:17:56.063403 17632 raft_consensus.cc:2243] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.067983 17632 raft_consensus.cc:2272] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.069597 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 9b37b6683af04823b0814ba56c4ea803 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:56.077280 17632 ts_tablet_manager.cc:1916] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:56.126505 17632 ts_tablet_manager.cc:1929] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.9
I20260709 13:17:56.127454 17632 log.cc:1199] T 74bea11400ce497283dceb7d21b6e728 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/74bea11400ce497283dceb7d21b6e728
I20260709 13:17:56.129079 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 74bea11400ce497283dceb7d21b6e728 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:17:56.137048 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:48200
I20260709 13:17:56.137735 11396 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:56.141307 11476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:17:56.141935 11476 raft_consensus.cc:606] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:17:56.143575 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:56.145372 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:56.369294 17434 raft_consensus.cc:993] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:56.369817 17434 raft_consensus.cc:1081] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:17:56.371807 12112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55066
I20260709 13:17:56.372336 12112 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:56.372680 12112 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.401764 12112 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:56.404975 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:17:56.405113 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:56.405619 11396 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:56.405802 11477 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.405998 11396 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:56.413677 11477 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:17:56.406839 11396 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:56.425983 11396 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.428282 12112 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:56.439234 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:56.439858 11546 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.473313 11546 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:17:56.476748 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:17:56.478461 17489 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:56.479036 17489 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:17:56.479969 17489 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:56.491515 11396 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:17:56.508827 11296 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.526243 17602 raft_consensus.cc:993] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:56.526859 17602 raft_consensus.cc:1081] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:17:56.531260 11396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:17:56.532083 11396 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:56.532526 11396 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:56.569092 11396 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:56.582886 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:17:56.583503 11477 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:56.583793 11477 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:17:56.584618 11477 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:56.585661 11477 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:56.594635 11477 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:56.595238 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:17:56.595851 11365 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:17:56.597249 17580 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:56.597728 17580 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:17:56.598565 17580 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:17:56.599566 11396 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:56.595891 11546 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:56.655056 11546 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:17:56.685611 11296 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 14, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:56.900213 11395 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:56.902083 17489 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:17:57.043133 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 96845.8us
I20260709 13:17:57.051724 11476 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:57.058705 17489 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:57.083757 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 139609us
I20260709 13:17:57.134514 11546 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:17:57.146474 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:57.159945 11546 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:57.154893 17573 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:17:57.162797 17580 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:17:57.167233 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 119257us
I20260709 13:17:57.182286 11477 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:57.186779 17580 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 13:17:57.312692 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 75246.9us
W20260709 13:17:57.402026 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 163843us
W20260709 13:17:57.413765 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50695.7us
I20260709 13:17:57.757668 11545 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:57.819036 11476 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:17:57.819221 11515 consensus_peers.cc:597] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 840a1b524ac54acdb12289fa1aafbd17. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.821972 17565 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:57.822221 11393 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:17:57.825186 17645 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:57.853911 17642 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:57.888547 11396 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:57.938898 11543 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:57.956017 11298 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.997679 11470 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.022480 11476 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:58.030679 17554 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:58.035228 11401 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:17:58.037045 11515 consensus_peers.cc:597] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25be57d1c24d4a43859ce0b5848adc09. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.038694 17645 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:58.111752 17642 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.122359 11476 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.144640 11402 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.152868 11396 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:58.202463 11476 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:58.203655 11545 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:58.205598 17659 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:58.212684 17225 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:17:58.216075 11366 consensus_peers.cc:597] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5a7fd6f89ac4da3ab7769d1379f5bb8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.251267 17657 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.254927 11471 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.277617 11545 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.281590 11549 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:58.302510 11296 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:58.342397 11293 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index -1 to 12, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.370496 11396 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:58.372228 17565 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:17:58.374305 11515 consensus_peers.cc:597] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e67029edeb04a569eb2c35676ef78da. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.369086 11471 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:58.377853 17554 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:58.457772 17676 ts_tablet_manager.cc:933] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:58.466810 17676 tablet_copy_client.cc:323] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:58.480347 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 840a1b524ac54acdb12289fa1aafbd17 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:58.481040 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 840a1b524ac54acdb12289fa1aafbd17 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-840a1b524ac54acdb12289fa1aafbd17
I20260709 13:17:58.509706 11561 tablet_copy_source_session.cc:215] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:58.514093 17565 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.514869 17676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840a1b524ac54acdb12289fa1aafbd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.520699 11470 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.514783 17678 ts_tablet_manager.cc:933] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:17:58.530575 17678 tablet_copy_client.cc:323] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:58.535552 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 25be57d1c24d4a43859ce0b5848adc09 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:58.537384 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 25be57d1c24d4a43859ce0b5848adc09 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-25be57d1c24d4a43859ce0b5848adc09
I20260709 13:17:58.545624 11396 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.620887 17676 tablet_copy_client.cc:806] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:58.624326 17676 tablet_copy_client.cc:670] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:58.667846 11293 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 13, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:17:58.709313 11561 tablet_copy_source_session.cc:215] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:58.711725 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:58.797399 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 144953us
I20260709 13:17:58.800442 17678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be57d1c24d4a43859ce0b5848adc09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.808671 17676 tablet_copy_client.cc:538] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:58.824891 17676 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
W20260709 13:17:58.839926 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 163781us
I20260709 13:17:58.846833 17678 tablet_copy_client.cc:806] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:58.847811 17678 tablet_copy_client.cc:670] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:58.858335 12122 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:17:58.888108 17678 tablet_copy_client.cc:538] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:58.891530 11471 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:58.893452 11546 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:58.895278 17656 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:58.898541 17678 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:58.900178 17647 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:59.028098 17690 ts_tablet_manager.cc:933] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
W20260709 13:17:59.124782 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 75252.1us
I20260709 13:17:59.106671 11396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:48200
W20260709 13:17:59.135646 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 91699.6us
I20260709 13:17:59.135186 11396 raft_consensus.cc:606] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:17:59.152165 17699 ts_tablet_manager.cc:933] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:17:59.154688 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 47286us
I20260709 13:17:59.145155 17690 tablet_copy_client.cc:323] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:17:59.176155 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:59.187567 17694 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:59.180028 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:59.207980 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:17:59.208657 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-c5a7fd6f89ac4da3ab7769d1379f5bb8
I20260709 13:17:59.217413 11471 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:59.218938 11412 tablet_copy_source_session.cc:215] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.224884 17690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:59.275774 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 128750us
I20260709 13:17:59.275792 11543 raft_consensus.cc:2955] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 3 to 9, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:59.297499 11298 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 3 to 9, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:17:59.305915 17699 tablet_copy_client.cc:323] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.3:46781
I20260709 13:17:59.312757 11561 tablet_copy_service.cc:140] P 9e9c2603c0bd4d25b713649b3890c46b: Received BeginTabletCopySession request for tablet 4e67029edeb04a569eb2c35676ef78da from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:41380)
I20260709 13:17:59.313315 11561 tablet_copy_service.cc:161] P 9e9c2603c0bd4d25b713649b3890c46b: Beginning new tablet copy session on tablet 4e67029edeb04a569eb2c35676ef78da from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:41380: session id = b3abf4b5ee724203ad1226f5c46bfe46-4e67029edeb04a569eb2c35676ef78da
I20260709 13:17:59.325731 11285 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 13:17:59.339944 11561 tablet_copy_source_session.cc:215] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.345172 17676 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:59.345790 17699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e67029edeb04a569eb2c35676ef78da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.347004 17676 tablet_bootstrap.cc:492] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:59.348137 17676 ts_tablet_manager.cc:1403] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.523s	user 0.054s	sys 0.035s
I20260709 13:17:59.351717 17676 raft_consensus.cc:359] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.352663 17676 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:59.357764 17676 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.361104 17676 ts_tablet_manager.cc:1434] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.005s
I20260709 13:17:59.362869 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-840a1b524ac54acdb12289fa1aafbd17 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:59.363400 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-840a1b524ac54acdb12289fa1aafbd17 on tablet 840a1b524ac54acdb12289fa1aafbd17 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:59.380750 17699 tablet_copy_client.cc:806] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.381480 17699 tablet_copy_client.cc:670] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:59.385277 17699 tablet_copy_client.cc:538] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:59.386660 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41722
I20260709 13:17:59.396377 17699 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:59.429590 17690 tablet_copy_client.cc:806] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.430461 17690 tablet_copy_client.cc:670] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
W20260709 13:17:59.431067 17678 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 13:17:59.457244 17690 tablet_copy_client.cc:538] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:59.485214 17699 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:59.486286 17699 tablet_bootstrap.cc:492] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:59.487092 17699 ts_tablet_manager.cc:1403] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.091s	user 0.044s	sys 0.016s
I20260709 13:17:59.509002 17678 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:59.572759 17690 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
I20260709 13:17:59.582930 17678 tablet_bootstrap.cc:492] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:17:59.587013 17678 ts_tablet_manager.cc:1403] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.689s	user 0.058s	sys 0.013s
I20260709 13:17:59.605765 17712 tablet_replica.cc:333] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:17:59.609325 17678 raft_consensus.cc:359] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.614362 17678 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:59.614715 17699 raft_consensus.cc:359] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.616472 17699 raft_consensus.cc:740] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:17:59.617877 17712 raft_consensus.cc:2243] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:59.620338 17712 raft_consensus.cc:2272] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:59.617470 17699 consensus_queue.cc:260] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.624866 17699 ts_tablet_manager.cc:1434] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.136s	user 0.005s	sys 0.006s
I20260709 13:17:59.634647 17712 ts_tablet_manager.cc:1916] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:59.693905 17659 raft_consensus.cc:993] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:59.694454 17659 raft_consensus.cc:1081] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:17:59.615402 17678 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:17:59.732600 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-4e67029edeb04a569eb2c35676ef78da received from {username='slave'} at 127.0.0.1:41380
I20260709 13:17:59.733105 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-4e67029edeb04a569eb2c35676ef78da on tablet 4e67029edeb04a569eb2c35676ef78da with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:17:59.715704 11471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43630
I20260709 13:17:59.755479 11471 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:59.756176 11471 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:59.753037 12122 raft_consensus.cc:1217] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEARNER]: Deduplicated request from leader. Original: 9.7->[9.8-9.8]   Dedup: 9.8->[]
W20260709 13:17:59.784408 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 66313.4us
W20260709 13:17:59.860318 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110484us
W20260709 13:17:59.869177 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 63891.5us
I20260709 13:17:59.920647 11471 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:17:59.983258 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 108984us
I20260709 13:17:59.992796 17678 ts_tablet_manager.cc:1434] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.405s	user 0.008s	sys 0.003s
W20260709 13:18:00.000424 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 112883us
W20260709 13:18:00.010757 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71689.4us
I20260709 13:18:00.016482 11471 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:00.130370 17712 ts_tablet_manager.cc:1929] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.8
I20260709 13:18:00.131201 17712 log.cc:1199] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/12c14d0fd7ef4fbf899a4b3eebe33e5b
W20260709 13:18:00.149149 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 133031us
W20260709 13:18:00.208989 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 185449us
I20260709 13:18:00.211328 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 12c14d0fd7ef4fbf899a4b3eebe33e5b (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:00.216964 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:00.217559 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 216232us
I20260709 13:18:00.224586 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:00.225294 11397 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:00.225584 11397 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:00.228264 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:00.229080 11549 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:00.226336 11397 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:00.233063 11397 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:00.229004 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 226246us
W20260709 13:18:00.242007 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 219996us
W20260709 13:18:00.243681 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 330618us
W20260709 13:18:00.250854 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 227622us
I20260709 13:18:00.255172 11397 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:00.257027 11477 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:00.251792 11549 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:00.262571 12123 raft_consensus.cc:1217] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.11->[14.12-14.12]   Dedup: 14.12->[]
I20260709 13:18:00.269382 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:00.271036 17727 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:00.271528 17727 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:00.272468 17727 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:00.327432 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50102.7us
I20260709 13:18:00.341631 12123 raft_consensus.cc:1217] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.12->[18.13-18.13]   Dedup: 18.13->[]
I20260709 13:18:00.367143 11561 tablet_copy_service.cc:342] P 9e9c2603c0bd4d25b713649b3890c46b: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-25be57d1c24d4a43859ce0b5848adc09 received from {username='slave'} at 127.0.0.1:41380
I20260709 13:18:00.367663 11561 tablet_copy_service.cc:434] P 9e9c2603c0bd4d25b713649b3890c46b: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-25be57d1c24d4a43859ce0b5848adc09 on tablet 25be57d1c24d4a43859ce0b5848adc09 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:00.468686 11298 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 16 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.494865 17690 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:00.501083 17690 tablet_bootstrap.cc:492] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:18:00.501943 17690 ts_tablet_manager.cc:1403] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 0.929s	user 0.044s	sys 0.021s
I20260709 13:18:00.510777 17690 raft_consensus.cc:359] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:00.511835 17690 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:18:00.512578 17690 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:00.519232 17690 ts_tablet_manager.cc:1434] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:18:00.625267 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-c5a7fd6f89ac4da3ab7769d1379f5bb8 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:18:00.625778 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-c5a7fd6f89ac4da3ab7769d1379f5bb8 on tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:00.644615 17741 raft_consensus.cc:1064] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:18:00.647223 17741 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:00.651871 17703 raft_consensus.cc:1064] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:18:00.670089 12123 raft_consensus.cc:1217] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.11->[17.12-17.12]   Dedup: 17.12->[]
I20260709 13:18:00.792446 17705 raft_consensus.cc:1064] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
W20260709 13:18:00.796806 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 108594us
I20260709 13:18:00.819095 17705 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:00.863407 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 50022.7us
W20260709 13:18:00.879395 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 47667.2us
I20260709 13:18:00.885316 17703 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:00.937675 11395 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:00.937825 12119 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:00.941864 17743 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:00.962728 17750 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:00.975330 11397 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:00.977470 17753 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:00.984115 11397 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:00.992514 11469 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:00.995549 11475 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:18:01.005328 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 44340.1us
I20260709 13:18:01.014101 12122 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:01.016018 11546 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:01.026660 17727 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:01.007368 17751 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:18:01.037448 17754 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:01.022296 17682 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:18:01.051146 17753 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:01.098907 12123 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEARNER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:01.123828 11395 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:01.129678 11475 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:01.125748 17565 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:18:01.138824 17733 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:01.143126 17741 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:18:01.151511 17565 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Committing config change with OpId 9.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.214869 17659 raft_consensus.cc:1064] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:18:01.269647 17659 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:01.290544 17752 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.299096 12115 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.323880 12122 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.334947 11395 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.345713 11469 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.371591 11393 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.336130 17751 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 13 to 14, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.395665 11293 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.483019 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:01.551746 11395 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:18:01.560644 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 106161us
I20260709 13:18:01.571885 11477 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:01.574460 12112 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:01.590780 11549 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:18:01.645378 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 56933.1us
I20260709 13:18:01.603861 17737 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:18:01.671046 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 57443.1us
I20260709 13:18:01.657074 12112 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.674376 11475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:01.737283 11475 raft_consensus.cc:606] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:01.741101 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:18:01.742877 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:01.686750 17659 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
W20260709 13:18:01.760293 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 98855.3us
W20260709 13:18:01.760703 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 102124us
W20260709 13:18:01.781129 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 118679us
I20260709 13:18:01.787654 11471 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.824409 17658 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:01.868618 11293 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.871233 11293 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 13 to 14, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } attrs { replace: true } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.915488 11475 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } attrs { replace: true } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.944263 17785 raft_consensus.cc:993] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:18:01.945137 17721 raft_consensus.cc:1081] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:18:01.956955 11543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:18:01.957523 11543 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:01.957866 11543 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:01.964920 11546 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:01.966385 11471 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.969581 11549 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:01.964993 17738 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:01.971518 11543 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:01.987962 11543 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:01.989882 11395 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:01.990217 12122 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:01.992703 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:01.992451 11549 consensus_queue.cc:237] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:01.993201 12122 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:01.994021 12112 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:01.997100 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:01.997766 11471 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:01.998119 11471 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:01.998826 17780 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:01.998919 11471 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:02.000437 11471 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:01.999470 11294 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.007477 11471 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:02.008991 11514 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:02.010768 17753 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:02.010560 11547 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.012756 17753 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:02.014539 17753 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:02.015573 17753 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:02.016431 17753 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:02.025404 11475 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:02.027225 17780 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:02.029300 17754 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Committing config change with OpId 9.10: config changed from index 9 to 10, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.043561 11393 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Committing config change with OpId 9.10: config changed from index 9 to 10, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.052259 12122 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:02.063557 12122 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:02.065173 17750 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.076262 11393 raft_consensus.cc:1275] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:02.079087 11298 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 9 to 10, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 9 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.080533 12122 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.084332 12112 raft_consensus.cc:2955] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Committing config change with OpId 9.10: config changed from index 9 to 10, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.090905 17753 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:02.095597 12123 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.097205 11293 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:02.104216 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 840a1b524ac54acdb12289fa1aafbd17 with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:59884
I20260709 13:18:02.104403 11393 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:02.106664 17751 consensus_queue.cc:1048] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:02.122550 17757 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Committing config change with OpId 18.15: config changed from index 14 to 15, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:18:02.136998 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 59836.7us
I20260709 13:18:02.136946 12122 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 14 to 15, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.141654 11471 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:02.143158 17738 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:02.146572 11396 raft_consensus.cc:2955] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 14 to 15, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.151518 17788 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.153127 11293 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.157101 11471 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.157914 11469 raft_consensus.cc:2955] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER 147201d22a694ca59f6e014f083a0892 (127.10.3.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.169621 11294 catalog_manager.cc:5697] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index 14 to 15, VOTER 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) evicted. New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.171027 12122 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:02.172329 17659 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:02.176333 11384 tablet_service.cc:1558] Processing DeleteTablet for tablet 25be57d1c24d4a43859ce0b5848adc09 with delete_type TABLET_DATA_TOMBSTONED (TS 147201d22a694ca59f6e014f083a0892 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:41722
I20260709 13:18:02.185096 12112 raft_consensus.cc:2955] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.187654 17795 tablet_replica.cc:333] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:02.188496 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 840a1b524ac54acdb12289fa1aafbd17 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:18:02.233219 11286 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44736.8us
I20260709 13:18:02.234206 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:02.234853 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 25be57d1c24d4a43859ce0b5848adc09 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:02.219568 11461 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e67029edeb04a569eb2c35676ef78da with delete_type TABLET_DATA_TOMBSTONED (TS 34fe1737ebb840fb9746980f649cc0a6 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:59884
I20260709 13:18:02.256385 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e67029edeb04a569eb2c35676ef78da with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:02.261798 11298 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: config changed from index 13 to 14, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:02.327440 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:51266
I20260709 13:18:02.342978 17797 tablet_replica.cc:333] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:02.348794 17796 tablet_replica.cc:333] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:02.350265 17796 raft_consensus.cc:2243] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:02.386533 17797 raft_consensus.cc:2243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:02.387562 17796 raft_consensus.cc:2272] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:02.387553 17797 raft_consensus.cc:2272] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:02.392354 17797 ts_tablet_manager.cc:1916] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:02.394312 17796 ts_tablet_manager.cc:1916] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:02.398366 17795 raft_consensus.cc:2243] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:02.400508 17795 raft_consensus.cc:2272] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:02.403846 17795 ts_tablet_manager.cc:1916] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:02.444314 17795 ts_tablet_manager.cc:1929] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.9
I20260709 13:18:02.447197 17795 log.cc:1199] T 840a1b524ac54acdb12289fa1aafbd17 P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/840a1b524ac54acdb12289fa1aafbd17
I20260709 13:18:02.447980 17797 ts_tablet_manager.cc:1929] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.13
I20260709 13:18:02.448469 17797 log.cc:1199] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/c5a7fd6f89ac4da3ab7769d1379f5bb8
I20260709 13:18:02.449769 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 840a1b524ac54acdb12289fa1aafbd17 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:02.450868 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet c5a7fd6f89ac4da3ab7769d1379f5bb8 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:02.451459 17795 tablet_replica.cc:333] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:02.452538 17795 raft_consensus.cc:2243] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:02.453394 17795 raft_consensus.cc:2272] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:02.456949 17796 ts_tablet_manager.cc:1929] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.13
I20260709 13:18:02.457685 17796 log.cc:1199] T 25be57d1c24d4a43859ce0b5848adc09 P 147201d22a694ca59f6e014f083a0892: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-0-root/wals/25be57d1c24d4a43859ce0b5848adc09
I20260709 13:18:02.457288 17795 ts_tablet_manager.cc:1916] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:02.459633 11286 catalog_manager.cc:5028] TS 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): tablet 25be57d1c24d4a43859ce0b5848adc09 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:02.512898 17795 ts_tablet_manager.cc:1929] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.14
I20260709 13:18:02.513481 17795 log.cc:1199] T 4e67029edeb04a569eb2c35676ef78da P 34fe1737ebb840fb9746980f649cc0a6: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-1-root/wals/4e67029edeb04a569eb2c35676ef78da
I20260709 13:18:02.515872 11284 catalog_manager.cc:5028] TS 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): tablet 4e67029edeb04a569eb2c35676ef78da (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:02.597826 12123 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:02.603446 17754 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:02.649466 11471 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:02.670336 17757 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:02.743831 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:02.860461 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 65618.4us
W20260709 13:18:03.208029 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 87086.4us
W20260709 13:18:03.231802 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 96844.1us
I20260709 13:18:03.500392 11393 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
W20260709 13:18:03.548090 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49122.3us
I20260709 13:18:03.563246 11546 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:03.565961 11477 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:03.575003 17716 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:03.577093 17716 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:18:03.601053 11366 consensus_peers.cc:597] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 -> Peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993): Couldn't send request to peer b3abf4b5ee724203ad1226f5c46bfe46. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c4eb2d16aa247998d0927eb34dedfa8. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:03.608320 17808 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:03.853281 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:03.885862 17738 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:18:03.950969 11293 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.963861 11543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3fd80658464474dbe463b1943becc75"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:18:03.966092 11543 raft_consensus.cc:606] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Received request to transfer leadership to TS 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:03.967597 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:18:03.969218 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:03.982642 11475 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:18:03.988065 11543 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } } }
I20260709 13:18:04.063356 17818 ts_tablet_manager.cc:933] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Initiating tablet copy from peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459)
I20260709 13:18:04.096127 17818 tablet_copy_client.cc:323] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Beginning tablet copy session from remote peer at address 127.10.3.1:37459
I20260709 13:18:04.098491 11412 tablet_copy_service.cc:140] P 147201d22a694ca59f6e014f083a0892: Received BeginTabletCopySession request for tablet 1c4eb2d16aa247998d0927eb34dedfa8 from peer b3abf4b5ee724203ad1226f5c46bfe46 ({username='slave'} at 127.0.0.1:50180)
I20260709 13:18:04.099140 11412 tablet_copy_service.cc:161] P 147201d22a694ca59f6e014f083a0892: Beginning new tablet copy session on tablet 1c4eb2d16aa247998d0927eb34dedfa8 from peer b3abf4b5ee724203ad1226f5c46bfe46 at {username='slave'} at 127.0.0.1:50180: session id = b3abf4b5ee724203ad1226f5c46bfe46-1c4eb2d16aa247998d0927eb34dedfa8
I20260709 13:18:04.115496 11412 tablet_copy_source_session.cc:215] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:04.121733 17818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c4eb2d16aa247998d0927eb34dedfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:04.166831 17818 tablet_copy_client.cc:806] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:04.167639 17818 tablet_copy_client.cc:670] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:04.171941 17818 tablet_copy_client.cc:538] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:04.457643 17818 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap starting.
W20260709 13:18:05.154429 17818 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.006s	sys 0.002s
W20260709 13:18:05.155030 17818 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.002s
W20260709 13:18:05.278334 11442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.002s	sys 0.001s
W20260709 13:18:05.278899 11442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.002s	sys 0.001s
I20260709 13:18:05.293287 11393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 3237) took 1073 ms. Trace:
I20260709 13:18:05.293983 11393 rpcz_store.cc:276] 0709 13:18:04.220089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.292424 (+1072335us) service_pool.cc:224] Handling call
0709 13:18:05.292774 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.292951 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.292978 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.293018 (+    40us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:18:05.293047 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.293101 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.293223 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.298126 11395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 3236) took 1078 ms. Trace:
I20260709 13:18:05.298673 11395 rpcz_store.cc:276] 0709 13:18:04.219730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.297275 (+1077545us) service_pool.cc:224] Handling call
0709 13:18:05.297626 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.297794 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.297817 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.297859 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:05.297888 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.297947 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.298064 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.314441 17826 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.324342 17826 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.330857 17833 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.333197 17834 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.333801 17825 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.343116 17833 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.345279 17833 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.345956 17834 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.348181 17834 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.348639 17826 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.349645 17825 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.351722 11397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50180 (request call id 1538) took 1072 ms. Trace:
I20260709 13:18:05.352458 11397 rpcz_store.cc:276] 0709 13:18:04.279397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.350562 (+1071165us) service_pool.cc:224] Handling call
0709 13:18:05.350975 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.351238 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.351277 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:05.351340 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:18:05.351413 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.351492 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.351641 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.351814 17825 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.375715 17839 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.390136 17839 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.393368 17839 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.396071 17832 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.396775 17832 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.398437 17825 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.399058 17825 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.399397 17832 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.400379 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:05.401156 17825 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.425880 17830 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.426707 17830 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.429585 17830 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.430608 17831 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.431216 17831 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.433445 17831 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.437269 17834 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.438031 17834 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.441655 17834 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.443316 17834 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.444044 17834 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.447938 17834 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.459177 17834 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:05.459620 11514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.197s	user 0.003s	sys 0.012s
W20260709 13:18:05.460007 11514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.004s	sys 0.012s
I20260709 13:18:05.459758 17834 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.461889 17834 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.473578 17848 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.476673 17848 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.491559 17848 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:05.499871 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 64884.7us
I20260709 13:18:05.506268 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:05.473632 17849 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.514248 17831 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.525381 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:05.527343 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:05.513554 17848 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.508074 17820 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.517999 17831 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.533427 17834 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.528630 17847 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.535892 17845 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.517539 17849 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.534302 17848 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.597587 17855 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.582360 17827 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.581944 17830 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.583937 17832 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.536079 17820 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.612176 17858 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.612960 17858 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.616770 17858 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.584339 17847 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.598178 17855 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.619753 17830 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.632774 17847 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.632997 17855 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.633262 17830 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.593142 17845 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.633344 17847 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.582006 17834 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.597522 17862 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.647496 17861 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.646629 17847 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.652280 17848 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.611570 17827 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.620935 17832 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.653498 17831 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.653525 17837 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.656153 17818 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:05.676424 17863 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.730254 17862 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.853955 17828 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.856618 17860 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.858207 17866 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.861392 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:05.862387 17853 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.866461 17841 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.650522 17846 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.870175 17867 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.899070 17864 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.899207 17870 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.900271 17865 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.900665 17857 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.870925 17849 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.650403 17852 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.872083 17861 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.873176 17851 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.888291 12122 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:05.876344 17826 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.896004 17872 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.909735 17844 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.897720 17871 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.910941 17845 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.900799 17830 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.911525 17845 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.911628 17830 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.912129 17861 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.912045 17845 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.912658 17861 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.913126 17861 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.913712 17830 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.914039 17845 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.914211 17830 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.914726 17830 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.915346 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:05.907876 17869 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.908118 17854 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.908257 17873 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.917433 17876 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.906651 17858 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.916679 17874 raft_consensus.cc:993] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:18:05.916756 17830 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.910267 17818 tablet_bootstrap.cc:492] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Bootstrap complete.
I20260709 13:18:05.925715 17875 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.906136 17855 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.930259 17845 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.922333 17826 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.934500 17832 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.934609 17824 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.935103 17874 raft_consensus.cc:1081] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Signalling peer 34fe1737ebb840fb9746980f649cc0a6 to start an election
I20260709 13:18:05.935240 17830 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.935837 17877 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.908758 17848 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.978914 17848 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.981037 17848 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.981195 17833 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.949249 17839 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.981756 17833 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.982052 17839 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.984045 17839 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.985265 17834 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.985795 17834 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.986260 17834 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.991834 17827 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.992460 17827 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.992930 17827 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.946341 17855 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.947932 17882 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:05.927021 17861 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.927649 17849 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:05.962394 11441 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:05.963985 17885 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.966569 17888 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.964188 17878 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:05.969626 17818 ts_tablet_manager.cc:1403] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent bootstrapping tablet: real 1.506s	user 0.044s	sys 0.013s
I20260709 13:18:05.975489 17743 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.975816 17830 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.976462 17826 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.977118 17862 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.977990 17835 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.952926 17845 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.978415 17859 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.924435 17858 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.003747 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.028692 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 110955us
I20260709 13:18:06.000900 17849 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.997004 17855 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.997625 17847 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.004504 17868 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:05.996335 17833 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.994963 17827 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.043164 17842 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.026266 17834 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.025516 17820 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.049299 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.080564 17848 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.039232 17832 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.083129 17848 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.083371 17832 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.085223 17832 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.086524 17830 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.086804 17845 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.088037 17881 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.085266 17848 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.088160 17849 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.080021 17862 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.069461 17855 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.071797 17868 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:05.899782 17870 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.923123 17876 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.107062 17874 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.106884 17862 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.033142 17885 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:06.118631 17876 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.891065 17860 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.899637 17864 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.119510 17862 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.119616 17885 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.977240 17877 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.882175 17837 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.936410 17875 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.120682 17860 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.121845 17877 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.917724 17873 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.122350 17837 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.122440 17875 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.910176 17872 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.891849 17828 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.124203 17873 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.898306 17867 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.124624 17872 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.995136 17824 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.125103 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.125735 12119 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:05.916188 17869 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.034417 17743 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.000475 17882 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.034178 17835 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.097281 17830 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.032914 17888 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.114424 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.184499 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.184929 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.185963 11545 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:06.186468 17888 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:06.187127 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71802.8us
I20260709 13:18:06.187384 17835 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.189658 17869 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:05.886571 17863 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.190974 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.192183 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.192345 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.192634 17863 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:05.921341 17866 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.117570 17890 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.117301 17874 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.910907 17871 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.910257 17844 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.905637 17846 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.035092 17859 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.920181 17851 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.069090 17826 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.093201 17832 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.126359 17867 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.169821 17824 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:05.919670 17852 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.169853 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:05.900842 17865 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:06.127702 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.038255 17878 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:05.916922 17854 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.171505 17870 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.171892 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.172839 11441 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.174826 17743 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.127512 11547 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 6.
I20260709 13:18:06.096017 17842 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.125149 17828 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.114560 17855 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.114953 17868 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:05.920476 17853 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.180469 17880 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.181403 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.181905 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:06.182058 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 71963.7us
I20260709 13:18:06.183353 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.184101 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.184590 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:05.922497 17841 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:05.901243 17857 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.088672 17881 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.259951 11549 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:18:06.200313 17858 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.260847 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.262319 17854 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.262514 17874 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.255162 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.264114 17890 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.266068 17859 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.266336 17855 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.225147 17888 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.267107 17890 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.186906 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.267704 17888 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.268489 17846 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.268857 17842 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.243731 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.185988 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.262598 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.260906 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.266655 17841 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.245779 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.372990 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.374362 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.375882 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.220360 17883 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.400743 17883 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.377126 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.183362 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.285276 17866 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.286435 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.402686 11550 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:18:06.120802 17864 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.378594 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.403522 17855 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.285728 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.379683 17865 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.380021 17853 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.380373 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.380569 17878 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.281587 17880 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.269979 12119 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
W20260709 13:18:06.288089 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 115433us
I20260709 13:18:06.310125 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:06.310359 10257 debug-util.cc:398] Leaking SignalData structure 0x7b0800158fe0 after lost signal to thread 11514
I20260709 13:18:06.382551 17852 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.295303 17826 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.288357 17743 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.246969 17818 raft_consensus.cc:359] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:06.405872 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.415421 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.405527 11468 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:18:06.416064 11546 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:06.245494 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.408766 17880 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.196792 17837 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.321547 17868 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.408949 17883 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.407312 17864 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.418321 17864 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.419405 17868 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.420037 11476 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:06.420329 17864 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.409981 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.421530 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.421906 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.422544 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.384521 17844 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.309713 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.423487 11393 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:06.423993 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.424321 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.425005 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.425374 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.425989 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.426409 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.426551 11476 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:18:06.286171 17871 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.285355 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.422928 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.427583 11469 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:18:06.427711 11472 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:06.428429 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.281967 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.429397 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.429674 11477 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 8.
I20260709 13:18:06.429951 11469 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:06.430518 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.430688 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.431548 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.431677 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.432096 11477 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:18:06.432641 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.432861 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.433760 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.198792 17882 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.195145 17830 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:06.326834 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080037e8e0 after lost signal to thread 10255
I20260709 13:18:06.371688 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:06.435391 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080037ed20 after lost signal to thread 11334
I20260709 13:18:06.435847 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.435981 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:06.436647 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08003331e0 after lost signal to thread 11426
I20260709 13:18:06.436854 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.437834 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.292132 17881 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.412954 17743 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.438431 11473 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:18:06.414381 17818 raft_consensus.cc:740] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Initialized, Role: LEARNER
I20260709 13:18:06.336697 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.389353 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.434751 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.390501 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.439497 17818 consensus_queue.cc:260] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: NON_VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: true } }
I20260709 13:18:06.392088 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.403553 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.441483 11470 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:06.392727 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.393066 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.446938 17826 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.393751 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.126766 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.299769 17857 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:06.283754 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.302333 17851 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.404114 17855 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.404625 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.405174 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.451318 11547 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:06.416432 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.439486 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.451970 17855 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.433682 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.364199 17888 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.438553 17743 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.391026 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.452979 11398 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:18:06.454365 17837 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.302652 17832 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.442353 17818 ts_tablet_manager.cc:1434] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent starting tablet: real 0.415s	user 0.009s	sys 0.006s
W20260709 13:18:06.437803 12144 debug-util.cc:398] Leaking SignalData structure 0x7b0800307600 after lost signal to thread 11501
I20260709 13:18:06.294402 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.436570 11472 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:18:06.459892 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.460546 11394 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 20.
W20260709 13:18:06.460629 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65771.3us
W20260709 13:18:06.460960 12144 debug-util.cc:398] Leaking SignalData structure 0x7b0800276a40 after lost signal to thread 11575
I20260709 13:18:06.462020 11441 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:18:06.462237 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7580 after lost signal to thread 12147
I20260709 13:18:06.462030 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.463297 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.459944 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.464990 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.465332 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.465886 12123 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:06.466120 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.466696 12112 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:06.469587 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.466939 17743 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.479095 17837 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.480309 17743 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.472882 17830 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.412540 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.484411 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.472151 17851 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:06.392402 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:06.481813 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.482558 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.484982 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:06.488080 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 95754.2us
I20260709 13:18:06.485590 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.449589 11470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3fd80658464474dbe463b1943becc75"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
 from {username='slave'} at 127.0.0.1:43640
I20260709 13:18:06.546613 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.486848 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.487242 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.485864 11547 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:18:06.555262 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.538529 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.557310 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.590260 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.563434 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:06.545472 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.521934 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.591212 12117 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.591424 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.591814 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.589470 12112 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:06.492794 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.589365 12123 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:06.592836 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.593619 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.593915 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.594172 12114 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:06.594492 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:06.595656 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 49141.7us
I20260709 13:18:06.592854 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.594928 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.590209 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.597113 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.597712 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.562868 11470 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:06.555634 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.598318 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.598631 11470 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.524245 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:06.595006 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.491883 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.490392 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.596163 12083 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.522749 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.496088 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.500916 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.600601 12083 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.600971 11396 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:06.601843 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.546489 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.500716 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.603343 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.498378 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.521061 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.522647 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.565404 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.494741 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.569088 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.487475 17851 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.608002 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.520884 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.559499 17832 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.575610 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:06.608879 12118 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.576411 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.576860 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 178819us
I20260709 13:18:06.577111 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.611218 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:06.583225 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.491108 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:06.460559 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 52574.9us
I20260709 13:18:06.493014 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.592566 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.602840 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:06.605154 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 110568us
I20260709 13:18:06.605861 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.606446 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.601416 17870 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:06.609536 12116 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:18:06.613981 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:06.618904 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.617595 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.495836 17837 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:06.617484 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.624845 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.618813 11367 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:06.620440 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:06.607550 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.612331 17830 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.612577 17851 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:06.612506 17878 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:06.621829 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:06.622350 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.612691 17855 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:06.622926 17890 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:06.623869 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.623703 17870 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.629851 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:18:06.632200 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 49077.7us
I20260709 13:18:06.644797 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.634709 12117 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:06.639482 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:06.643184 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.620498 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.646970 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.618150 17856 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.649185 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.650609 11441 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.630353 17887 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:06.624383 17837 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.630686 17834 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.631062 17830 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.617825 17867 consensus_queue.cc:1059] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:18:06.667398 11441 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.672251 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:06.640046 17849 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.639703 17878 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.652066 17857 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.656001 17887 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.699596 17887 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:06.680398 17830 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.659821 17842 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.709190 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.711958 11547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:18:06.712599 11547 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:06.713194 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 45948.1us
I20260709 13:18:06.712142 12118 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:06.676694 11470 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.640832 17855 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.641039 17890 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.715796 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:18:06.646548 17870 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:06.680351 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 59927.1us
I20260709 13:18:06.723073 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.669715 17837 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.717418 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:06.728994 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 100430us
I20260709 13:18:06.729993 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:06.726300 11441 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.734236 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.724668 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:06.715543 17866 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.696910 17847 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:06.703538 17863 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.744382 17863 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.744961 17863 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.705361 17878 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:06.735308 11470 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.749547 11441 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.735852 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.731459 11367 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:06.718149 17857 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.743896 17866 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.744427 17898 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:06.705797 17841 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:06.717738 17890 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:06.715950 17855 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:06.708132 17842 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.729875 17831 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:06.754050 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.779817 11441 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.757161 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.782305 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 161913us
I20260709 13:18:06.777483 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.751757 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.751801 11367 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.751410 17867 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:06.757303 17866 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.780138 12123 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:06.798067 17895 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:06.798537 17895 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:06.798899 17895 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.806420 17833 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.807026 17833 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.808588 17866 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.809078 17833 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.809613 17868 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:06.810168 17868 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:06.780571 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.748437 17849 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.754127 17846 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:06.815496 17846 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.815900 17846 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:06.758551 17847 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:06.753863 17857 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.823220 17857 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.823578 17856 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:06.824517 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 44030.5us
I20260709 13:18:06.825910 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.756117 17894 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.827198 17894 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.827597 17894 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.829244 17895 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.830385 17856 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.794252 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.791585 17867 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:06.831584 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.810707 17866 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.831800 17867 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.832954 17867 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.833720 17895 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.834372 17856 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.786139 17860 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:06.835722 17860 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.836093 17860 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.837774 17896 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:06.790411 17891 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.789588 17842 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.838999 17891 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.836146 17883 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:06.843513 17883 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.843899 17883 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:06.846333 17826 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.846895 17826 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.848937 17826 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.852240 17883 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.853947 17863 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.814954 17849 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.851498 17860 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.854846 17849 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:06.856395 17860 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.856158 17887 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.858843 17846 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.860353 12123 raft_consensus.cc:1217] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:18:06.860798 12117 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:06.861604 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.861706 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.812453 11441 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:06.814360 17868 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.882746 17847 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:06.862826 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.864971 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.885931 17855 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.841156 17891 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.836059 11442 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:06.888348 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 76001.7us
I20260709 13:18:06.888674 17743 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.888794 17855 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.889443 17743 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:06.890026 17743 raft_consensus.cc:697] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:06.880246 17851 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.890897 17743 consensus_queue.cc:237] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.891888 17851 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.868541 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.858873 17887 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.862816 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.894519 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 102428us
I20260709 13:18:06.864348 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.882426 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 88278.4us
I20260709 13:18:06.896593 11515 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:06.897240 11516 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.898919 11515 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.854316 17883 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.900993 17847 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:06.901189 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:06.901485 17847 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:06.904729 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:06.901882 17847 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:06.907569 17883 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:06.907987 17883 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:06.908355 17883 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.909252 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.910053 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:06.914669 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:06.784951 17841 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
W20260709 13:18:06.897284 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 61277.3us
I20260709 13:18:06.922488 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:06.923209 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:06.934917 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.937053 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:06.952610 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 53773.4us
I20260709 13:18:06.957597 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:06.922539 17841 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:06.910368 17890 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:06.961565 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 209834us
I20260709 13:18:06.965097 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.936635 17820 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:06.965912 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:06.975903 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:06.954998 17884 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.910513 17878 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:06.978008 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:06.956203 17860 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:06.980212 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.982352 17892 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:06.982419 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:06.984642 17863 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:06.985935 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.987278 17830 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:06.989547 12116 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 17: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:06.990417 17830 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:06.964646 17853 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:07.001935 17837 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.050966 17837 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.027693 17853 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:07.052003 17853 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.054445 17841 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.030798 17870 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.056957 17870 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.010208 17849 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.038301 17860 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.059959 17849 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.060250 17860 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.030841 17890 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.039117 17883 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.039292 17857 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.062835 17883 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.062678 17847 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:07.042840 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 88126.3us
I20260709 13:18:07.063809 11551 raft_consensus.cc:1240] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:07.038192 11294 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 17 to 18, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 18 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.063174 11547 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:07.002406 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:06.994978 17894 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:07.007433 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 335222us
I20260709 13:18:07.068552 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.069029 17894 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.067384 17857 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:07.069610 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.069800 17857 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:07.070206 17857 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:07.070894 17883 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.070688 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.072705 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:07.044298 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 71469.1us
I20260709 13:18:07.074926 17883 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.075073 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.075636 17883 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.076242 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.077391 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.077601 17905 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.078202 17849 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.078353 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.051563 17878 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.080070 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.080379 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:06.916973 17842 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.048750 17846 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.070562 17857 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.083760 17842 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.086154 12119 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:07.087059 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 295379us
I20260709 13:18:07.088167 12116 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:07.089591 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.089813 17847 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.090302 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.073287 17853 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.090798 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.087060 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.097283 17883 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.096784 17853 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.100518 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.105875 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.107558 17883 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:07.109125 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 145264us
I20260709 13:18:07.110009 17868 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.111379 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.111702 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.112522 11514 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.115831 11550 raft_consensus.cc:1240] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.118368 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.112319 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.124166 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.125371 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.125969 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.126299 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:07.127971 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 162120us
I20260709 13:18:07.113823 17841 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:07.129603 11366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.4:45993 timed out after 1.544s (SENT)
I20260709 13:18:07.136592 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.136838 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.138175 11475 raft_consensus.cc:1240] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:07.139356 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.139617 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.139847 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.129704 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:07.126945 17883 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.132071 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.141652 11473 raft_consensus.cc:1240] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:07.141552 12085 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.142267 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.107550 17857 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.140985 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.126924 17898 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:18:07.141727 17849 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.147454 17851 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.147727 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.148706 17869 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:07.149775 17869 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:07.143750 12085 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.146901 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.123509 17868 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:07.144398 17830 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.121084 17894 consensus_queue.cc:1059] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:18:07.155488 17894 raft_consensus.cc:3055] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:07.155675 17857 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.155836 17894 raft_consensus.cc:740] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.151952 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.138981 17905 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.152663 17837 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:07.152108 17868 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.157789 17893 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.158670 17898 consensus_queue.cc:1059] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:07.159106 17868 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.153822 17878 consensus_queue.cc:1059] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:07.154186 11469 raft_consensus.cc:1240] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:07.153738 17841 consensus_queue.cc:1059] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:07.144687 17855 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:07.150135 17869 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:07.160614 17898 raft_consensus.cc:3055] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.164402 17898 raft_consensus.cc:740] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.159935 17837 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.164958 17905 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:07.165319 17837 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.165984 17905 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:07.166342 17905 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:07.163640 17855 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:07.167438 17855 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:07.167722 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.168205 17878 raft_consensus.cc:3055] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:07.168601 17878 raft_consensus.cc:740] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.168977 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.169391 17878 consensus_queue.cc:260] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.169948 17866 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.170612 17878 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:07.170715 17866 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.171335 17860 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.188129 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.171766 17869 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.189564 11400 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:07.189810 17869 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:07.191954 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.193661 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.196527 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.156661 17894 consensus_queue.cc:260] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.199508 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.200904 11399 raft_consensus.cc:1240] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.201118 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.174531 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.175150 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.177153 17831 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:07.166746 17837 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:07.167181 17905 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.180169 17898 consensus_queue.cc:260] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.181460 17855 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.189603 12085 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.183626 17890 raft_consensus.cc:3055] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:07.184911 11393 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.186681 11367 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.217074 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.202135 11401 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.199548 11471 raft_consensus.cc:1240] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.202342 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.213941 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:07.218969 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.182081 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.223476 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:07.226538 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.227124 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.229045 17898 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.231123 17857 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:07.231521 17857 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:07.231880 17857 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:07.233094 17855 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.211373 17831 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:07.231144 17874 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.233979 17831 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.234203 17884 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:07.235823 17847 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.235886 17830 consensus_queue.cc:1059] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:18:07.237481 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 107919us
I20260709 13:18:07.238906 11515 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.228225 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.241328 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.241341 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:07.243767 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.218086 17894 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:07.247391 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.218340 17890 raft_consensus.cc:740] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.217291 17867 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.252830 17890 consensus_queue.cc:260] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.244504 17893 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.254158 17890 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.248296 17899 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:07.255825 17899 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.225808 17906 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.222460 17856 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:07.242242 17874 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.258118 17856 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.226204 17827 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.226224 17883 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.260416 17835 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:07.260859 17835 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.262614 17909 consensus_queue.cc:1059] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:07.263722 17909 raft_consensus.cc:3055] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:07.264050 17909 raft_consensus.cc:740] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.265302 17909 consensus_queue.cc:260] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.266561 17909 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.244361 17868 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.215629 17900 consensus_queue.cc:1059] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:18:07.242594 17847 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.268214 17881 raft_consensus.cc:3055] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:07.268705 17881 raft_consensus.cc:740] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:07.222672 17841 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.269614 17881 consensus_queue.cc:260] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.247383 17884 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.268275 17900 consensus_queue.cc:1243] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer 34fe1737ebb840fb9746980f649cc0a6 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:18:07.270902 17881 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:07.279412 11366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.261600 17835 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.280786 17835 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:07.287214 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.287559 17841 raft_consensus.cc:3055] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.288005 17841 raft_consensus.cc:740] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.289106 17841 consensus_queue.cc:260] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.287868 11542 raft_consensus.cc:3055] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:07.290069 11542 raft_consensus.cc:740] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:07.290416 17841 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.247766 17905 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:07.290740 11542 consensus_queue.cc:260] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.291676 11542 raft_consensus.cc:3060] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.293834 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.294350 12119 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:07.295907 17852 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:07.299533 11397 raft_consensus.cc:2468] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:07.300907 11547 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:07.288048 11401 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:07.304715 11393 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.305706 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.306815 11542 raft_consensus.cc:2468] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:07.310771 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.312351 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.314102 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.315382 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.316268 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.316632 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:07.318742 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.322433 11396 raft_consensus.cc:1240] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:07.323375 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.324218 11551 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:07.326277 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.327106 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:07.327329 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.327644 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:07.328143 11412 tablet_copy_service.cc:342] P 147201d22a694ca59f6e014f083a0892: Request end of tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-1c4eb2d16aa247998d0927eb34dedfa8 received from {username='slave'} at 127.0.0.1:50180
I20260709 13:18:07.335480 17899 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.335656 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:07.333027 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.327106 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.336807 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.337385 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.338321 17890 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.333539 12117 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:07.334673 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.332999 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.335510 11412 tablet_copy_service.cc:434] P 147201d22a694ca59f6e014f083a0892: ending tablet copy session b3abf4b5ee724203ad1226f5c46bfe46-1c4eb2d16aa247998d0927eb34dedfa8 on tablet 1c4eb2d16aa247998d0927eb34dedfa8 with peer b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.342721 17856 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.345098 12117 raft_consensus.cc:1240] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:07.345736 17868 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:07.346122 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.349241 11476 raft_consensus.cc:1240] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:07.338243 11366 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.350095 11442 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:07.350327 17899 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:07.351784 11366 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.352749 11442 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.352921 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.353749 11550 raft_consensus.cc:2393] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 17: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:07.353799 17890 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.316979 17852 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:18:07.358484 17890 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.359184 17860 consensus_queue.cc:1059] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:07.360318 17860 raft_consensus.cc:3055] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:07.360965 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.361402 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:07.361358 11401 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:07.363842 11514 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.365053 17852 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:07.365456 17852 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.365818 17831 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.363020 17856 raft_consensus.cc:2804] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:07.360657 17860 raft_consensus.cc:740] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.367583 17856 raft_consensus.cc:697] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.366418 17852 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.331496 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.369050 17852 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:07.368920 17856 consensus_queue.cc:237] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.368010 17860 consensus_queue.cc:260] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.371622 17860 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:07.381558 12085 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.383458 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.384183 17890 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.389490 17893 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:07.389945 17893 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:07.390291 17893 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:07.345832 17857 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.394099 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:07.397791 11514 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.400489 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.402967 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:07.406615 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 53929.8us
I20260709 13:18:07.410609 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:07.412181 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.415469 12116 raft_consensus.cc:1240] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 20. Current term is 21. Ops: []
I20260709 13:18:07.415946 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.425870 11515 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.426849 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.430195 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.431885 12085 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.368063 17831 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:07.433259 17907 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:07.403679 17869 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.470345 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.409401 17894 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:07.435544 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.473645 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.434278 17882 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.471446 17896 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.470697 17907 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:07.464928 17899 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.475194 17907 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.471700 17870 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.479522 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.480528 17860 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:07.480970 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.466523 17824 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:07.481683 17824 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.482061 17866 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.482182 17832 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.482522 17824 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.483422 11395 raft_consensus.cc:1275] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:07.466600 17837 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:07.485759 17857 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:07.489658 17893 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.490931 11471 raft_consensus.cc:1240] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:07.496114 17834 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.494745 11547 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:18:07.411142 17868 raft_consensus.cc:2804] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:07.473855 12085 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.498605 11400 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
W20260709 13:18:07.471887 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 46098.3us
I20260709 13:18:07.410724 17883 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.506699 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.512279 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.514087 12085 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.508014 17832 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.528800 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.532537 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.514810 17855 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.514724 17830 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.514896 17899 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:07.557520 17893 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.535058 11442 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.537792 11395 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:18:07.558104 17852 consensus_queue.cc:1048] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:07.559892 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:07.560359 17852 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:07.561002 17852 raft_consensus.cc:697] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.514681 17868 raft_consensus.cc:697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.505973 17905 consensus_queue.cc:1059] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:18:07.563699 17905 raft_consensus.cc:3055] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:18:07.563737 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:07.564323 17905 raft_consensus.cc:740] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:07.545348 11547 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:07.535566 17881 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.565282 17905 consensus_queue.cc:260] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.566622 17905 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:07.527408 17841 consensus_queue.cc:1059] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:07.568024 17866 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:07.569406 17847 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.571350 17868 consensus_queue.cc:237] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.574220 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:07.576150 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 62142.2us
I20260709 13:18:07.576960 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.578938 17830 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:07.581931 17890 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:07.584802 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.584952 12085 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.595710 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.596709 17906 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:07.598074 12085 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.600421 12085 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.600960 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.582933 17890 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:07.602761 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.603152 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.604434 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.550164 17856 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.605530 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.605815 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.607987 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.609856 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.611060 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.611308 11400 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:07.609756 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:07.614251 11547 raft_consensus.cc:1240] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:07.604873 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:07.619305 11400 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.619657 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:07.597954 17870 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.602190 17906 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.603467 17871 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:07.603685 17866 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:07.603690 17890 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.629025 11298 catalog_manager.cc:5697] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 13 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:07.632943 11366 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.562079 17852 consensus_queue.cc:237] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.636971 11366 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.638064 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.603226 17855 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.633870 11401 raft_consensus.cc:1240] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:07.601791 17882 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:18:07.639449 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.647816 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.645105 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.647171 17900 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:07.646229 17894 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.675125 17837 raft_consensus.cc:3055] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:07.675657 17837 raft_consensus.cc:740] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:07.676503 17837 consensus_queue.cc:260] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.662165 17890 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:07.646052 17841 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:07.656091 17827 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:07.648550 17878 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.649163 17866 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:07.631322 17907 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.667073 17906 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:07.741552 12113 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:07.647612 17870 consensus_queue.cc:1059] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:18:07.698777 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:07.745481 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46799.4us
I20260709 13:18:07.746217 12085 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.739255 17907 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.667558 17893 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:18:07.749626 17917 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:07.751438 12085 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.741307 17906 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:07.756109 12085 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:07.749015 17830 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:07.753297 17890 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:07.756417 17882 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:07.759227 17907 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:07.763425 17907 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:07.764380 17907 raft_consensus.cc:3060] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:07.760454 17866 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.761951 17906 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.768293 17906 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.774436 17837 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:07.778101 17907 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.811141 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:07.744251 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:07.851552 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 93342.6us
I20260709 13:18:07.875653 17907 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:07.890657 17882 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:07.878095 17832 raft_consensus.cc:3055] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:07.904238 17830 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:07.904817 17830 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:07.900678 17832 raft_consensus.cc:740] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:07.930418 17832 consensus_queue.cc:260] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.932551 17832 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:07.916591 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.935420 17834 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.936364 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.939425 17878 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.906440 17830 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.944474 17830 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:07.893361 17882 consensus_queue.cc:1059] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:18:07.946913 17882 raft_consensus.cc:3055] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:07.947500 17882 raft_consensus.cc:740] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:07.949375 17882 consensus_queue.cc:260] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:07.926743 17894 raft_consensus.cc:1064] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: attempting to promote NON_VOTER b3abf4b5ee724203ad1226f5c46bfe46 to VOTER
I20260709 13:18:07.938851 11366 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.966303 17878 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:07.968281 11366 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:07.969252 17878 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:07.976462 11542 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:18:08.002396 11547 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:08.014735 17894 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.023826 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:08.026014 17834 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:08.038434 17835 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:08.052713 11398 raft_consensus.cc:1275] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:18:08.057181 11547 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:18:08.058511 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.063324 17851 consensus_queue.cc:1048] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:07.902290 17907 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.164s
I20260709 13:18:08.066498 17832 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:08.066923 17832 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.067958 17832 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.069798 17832 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:08.070447 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.072351 17882 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:08.085991 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:08.092633 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.094141 11542 raft_consensus.cc:2393] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 16: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:08.116973 17851 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:08.118577 17851 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:08.119051 17851 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:08.120288 17851 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.122499 17851 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.137687 17824 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:08.139953 17930 consensus_queue.cc:1059] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:08.141149 17832 raft_consensus.cc:3055] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:08.141582 17832 raft_consensus.cc:740] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.142067 12085 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.142648 17832 consensus_queue.cc:260] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.143185 17930 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:08.144345 17832 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.155956 11293 catalog_manager.cc:5697] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 17 to 18, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 18 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } }
I20260709 13:18:08.178951 11401 raft_consensus.cc:2468] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
W20260709 13:18:08.197332 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 55614.4us
I20260709 13:18:08.199133 17832 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:08.199311 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.200225 17930 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:08.200284 17832 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:08.200696 17930 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.201713 17930 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.204025 17930 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.226783 11401 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:08.235721 17824 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:08.245810 12119 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:18:08.238651 11298 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:18:08.271260 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72058.9us
I20260709 13:18:08.280143 12122 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:08.281982 12117 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:08.302531 11476 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:08.303058 12115 raft_consensus.cc:3060] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.308382 12112 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:08.314891 11397 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:08.315829 17837 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:08.316933 17909 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:08.317054 11294 catalog_manager.cc:5697] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 16 to 17, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.322451 17853 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:08.325866 17824 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:08.330090 17853 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:08.363821 12115 raft_consensus.cc:1275] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:08.363678 12117 raft_consensus.cc:1275] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:18:08.375954 17856 consensus_queue.cc:1048] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:08.386353 12085 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.389043 17835 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:08.396373 11547 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:08.416772 17930 raft_consensus.cc:2804] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:08.417661 17930 raft_consensus.cc:697] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.430101 11545 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:18:08.394899 17835 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
W20260709 13:18:08.431926 17936 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:08.432173 17835 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.433702 17835 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:08.428526 17930 consensus_queue.cc:237] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.451359 17890 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.451992 17890 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:08.387344 17934 log.cc:927] Time spent T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:08.462082 17743 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:18:08.467823 17890 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:08.469153 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.469861 11468 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
W20260709 13:18:08.473604 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 89900.4us
I20260709 13:18:08.475584 11365 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:08.481720 17878 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:08.482213 17878 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.482566 17878 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:08.484692 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:08.485437 11543 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:08.488329 17834 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:08.389369 17856 consensus_queue.cc:1048] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.114s
I20260709 13:18:08.516966 12112 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEARNER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:08.524922 11545 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:08.527077 11294 catalog_manager.cc:5697] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 14, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.543910 17832 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:18:08.543102 17834 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:18:08.550161 17824 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.542955 17878 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.588330 17878 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:08.587666 11298 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 14 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 15 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.570379 17832 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:08.589546 17832 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.590677 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.590699 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:08.591379 11545 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:08.591480 11468 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:18:08.595053 17824 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.590575 17832 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.597092 17832 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:08.597595 17824 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.605950 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.606231 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:08.606775 12122 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 7.
I20260709 13:18:08.610940 17834 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.612849 11543 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.626595 11545 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:08.628793 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.636631 17824 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:08.637048 17824 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.636965 11367 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:08.637565 17824 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:08.652197 17890 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:08.680204 11468 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:08.675923 17890 raft_consensus.cc:697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:08.708972 17890 consensus_queue.cc:237] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.743081 11476 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.746662 11298 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.747376 17824 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.760562 17930 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.761089 17930 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.763581 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.764298 11397 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:08.765554 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:08.765708 17930 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:08.766012 17951 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.766395 17930 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:08.766659 17951 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.768656 17930 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.769147 17930 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.769606 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:08.770363 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:08.773686 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:08.774578 17834 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:08.775053 17834 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.775420 17834 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.780082 17824 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.780738 17824 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.781216 17824 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.783269 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.789377 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.790017 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.790697 11476 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:08.790017 11397 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:08.794193 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:08.795073 17852 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:08.795509 17852 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.795861 17852 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:08.796576 17824 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.804039 17905 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.804688 17905 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.805306 17834 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.807653 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.808238 17820 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.808527 11397 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:08.808799 17820 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.809446 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.809558 17951 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.809981 11545 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.811508 17824 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.810890 17820 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.816169 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.816651 17847 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.816970 11471 raft_consensus.cc:2368] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
I20260709 13:18:08.817297 17847 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.817862 17834 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.807686 17905 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:08.812040 17824 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.819567 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.820253 11468 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:08.821007 17824 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:08.821400 17847 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:08.822803 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.823310 11514 leader_election.cc:400] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
I20260709 13:18:08.823560 11397 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 18: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:08.823604 11514 leader_election.cc:403] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.824361 17847 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:08.824844 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:08.825472 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.826202 11397 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:08.827121 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:08.827772 11367 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:08.828464 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:08.829754 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.830488 11468 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:18:08.829825 17834 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:08.831200 17820 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:08.831333 17834 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:08.831679 17820 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.832186 17820 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:08.832424 17834 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.841140 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.841459 17847 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 7. Current term is 8.
I20260709 13:18:08.842032 17824 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:08.842454 17824 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.842617 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:08.843829 17905 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:08.844311 17905 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.842800 17824 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.844712 17905 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:08.851991 17824 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.853996 17824 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:08.857509 17865 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.858089 17865 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.860487 17865 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.862160 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.876216 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:08.876811 11400 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.878014 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:08.881327 12115 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index 8 to 9, b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:08.884099 11400 raft_consensus.cc:2468] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:08.886900 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.887764 12116 raft_consensus.cc:2393] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 7: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:08.888919 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.889503 12117 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:08.889962 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:08.890558 12123 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.891736 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:08.892261 12122 raft_consensus.cc:3060] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:08.894433 17743 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:08.895013 17743 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.898041 17743 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.890645 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.901136 12119 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:18:08.901297 11400 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:08.904168 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:08.904215 17870 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:08.905108 17928 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:08.905551 17928 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.905871 17928 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:08.907224 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.908021 12116 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:18:08.909235 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:08.910010 17851 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:08.910369 17851 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:08.910725 17851 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:08.916410 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.919600 12123 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:08.913002 11293 catalog_manager.cc:5697] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 15 to 16, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 16 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.923343 17865 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:08.924507 17865 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.927145 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.927469 12122 raft_consensus.cc:2468] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 8.
I20260709 13:18:08.927726 17865 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:08.927824 11400 raft_consensus.cc:2368] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:08.928610 11441 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:08.931479 11516 leader_election.cc:400] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:08.931813 11516 leader_election.cc:403] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.933360 17824 raft_consensus.cc:2804] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:08.933979 17824 raft_consensus.cc:697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:08.935197 17865 raft_consensus.cc:3060] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.935847 17824 consensus_queue.cc:237] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.939908 17930 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.943090 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:08.943675 11400 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.950554 11400 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 19.
I20260709 13:18:08.951632 12085 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:08.952423 17870 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:08.952898 17870 raft_consensus.cc:697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:08.953814 17870 consensus_queue.cc:237] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.959111 17743 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:08.959967 17743 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.965148 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.970332 17743 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:08.971364 11400 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:08.972239 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:08.972932 12123 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:08.974320 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:08.975265 17743 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:08.975728 17743 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:08.976166 17743 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:08.922865 17851 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:08.985105 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:08.988571 17820 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:08.991417 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:08.992116 11400 raft_consensus.cc:2393] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:08.950263 17852 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.004842 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:08.990850 17820 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.019606 11400 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 19: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:09.008963 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:08.985543 17905 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.014284 17951 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.028219 17951 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.030529 17951 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.036665 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.037312 11468 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:09.038405 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:09.039260 17878 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:09.039650 17878 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.039986 17878 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:09.020978 11476 raft_consensus.cc:2368] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:09.024331 12123 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:18:09.026160 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.027592 17905 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:08.990288 17928 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.042423 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.046193 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.046816 11545 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:09.006085 17851 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.051611 11293 catalog_manager.cc:5697] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.052663 11442 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:09.052233 17853 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:09.084651 11514 leader_election.cc:400] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:09.087146 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.087615 11441 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.060904 17930 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.095989 17853 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.099130 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.099982 11395 raft_consensus.cc:2393] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 9: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:09.068239 17856 raft_consensus.cc:2804] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:09.104475 11471 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:09.114514 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.115208 11468 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:09.124871 11476 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:18:09.104045 17835 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:09.105984 17824 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.116361 17856 raft_consensus.cc:697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:09.136638 17743 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.125593 17865 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:09.139650 17852 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.128082 17890 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.140861 17890 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.143179 17743 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.145653 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.146430 11545 raft_consensus.cc:2368] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14.
I20260709 13:18:09.147701 11367 leader_election.cc:400] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14.
I20260709 13:18:09.147977 11367 leader_election.cc:403] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:09.148532 17951 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.151808 17928 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.154940 17878 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.133997 17835 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:09.156518 17835 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.159345 11400 raft_consensus.cc:2368] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
W20260709 13:18:09.160312 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 72774.6us
I20260709 13:18:09.161059 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
W20260709 13:18:09.164170 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 122409us
I20260709 13:18:09.165273 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:09.166550 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:09.167693 11545 raft_consensus.cc:1275] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:18:09.172180 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:09.172740 12122 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.175153 12123 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:09.181186 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:09.180440 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.183740 17743 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:09.184195 17743 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:09.190353 17856 consensus_queue.cc:237] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.192724 11516 leader_election.cc:400] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:09.193039 11516 leader_election.cc:403] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:18:09.193765 17896 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:09.193778 11469 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:18:09.198539 17882 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.199265 17882 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.205921 17894 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:09.206346 17882 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.211154 11468 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 19.
I20260709 13:18:09.212258 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.160362 17824 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.235097 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.231504 17743 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.235949 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.234759 12122 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:18:09.236961 11401 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:09.215024 11293 catalog_manager.cc:5697] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:09.241214 11515 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.231091 17890 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.241011 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.188998 17832 consensus_queue.cc:1048] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:18:09.256784 11545 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:09.266855 11545 raft_consensus.cc:1275] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:18:09.267244 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.267596 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.268163 11468 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:09.270741 11401 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:18:09.266491 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:09.274119 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.274690 11401 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:09.274819 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.275627 11400 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 19: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:09.256016 17847 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:09.294739 17878 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.295809 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.261413 17961 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:09.303581 17961 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.304069 17961 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:09.304646 11543 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:09.306155 17890 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:09.308450 17824 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.308888 11468 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:09.315734 17957 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.316036 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.316303 17957 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.317835 12115 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:09.319229 11365 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:09.285908 17820 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:09.281744 17834 consensus_queue.cc:1048] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:18:09.318183 17847 raft_consensus.cc:697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:09.355409 11401 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:18:09.379866 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:09.390002 17951 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 13. Current term is 14.
I20260709 13:18:09.399021 11442 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:09.401423 17743 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:09.395507 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 71694.5us
I20260709 13:18:09.402895 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.403636 11543 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:18:09.403986 17853 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.405192 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.379690 17865 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.403541 17969 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.407500 17743 raft_consensus.cc:697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:09.324528 17890 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:09.413259 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.436126 11469 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:18:09.401259 17856 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:09.406136 17957 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.438896 17856 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:09.438490 17837 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.438694 17896 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 18. Current term is 19.
I20260709 13:18:09.447595 17974 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:09.447317 17847 consensus_queue.cc:237] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.446187 17743 consensus_queue.cc:237] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.431175 17865 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.464989 17865 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.434918 17969 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.467716 17837 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.469105 17969 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.445075 17928 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:09.439581 17890 raft_consensus.cc:697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:09.470647 17837 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.514802 17820 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:09.478034 17974 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.524169 11395 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:09.445077 17856 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:09.494549 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.538193 11543 raft_consensus.cc:2393] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:09.553938 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.555289 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.559931 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.554661 12112 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 21.
I20260709 13:18:09.475829 17928 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.580904 11397 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:18:09.523042 17974 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:09.590425 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 43915.9us
I20260709 13:18:09.593163 11543 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:09.599304 11545 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:09.605440 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.579351 17977 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.579659 17962 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.592377 17837 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:09.625741 17820 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.626260 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.631649 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.632453 17870 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:18:09.645581 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 99626.6us
I20260709 13:18:09.620460 17852 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.616353 17961 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.657775 17852 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.659281 17961 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.643994 17977 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.660245 17852 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.661674 11550 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 15: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:09.661834 17977 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.636229 17951 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:09.652827 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 90308.4us
I20260709 13:18:09.675230 17951 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.667506 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.677325 17951 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.633793 17962 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.677750 11468 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:18:09.667775 17837 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.679622 17962 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.645159 17969 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.668491 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.680337 17969 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.680361 17837 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.682487 17969 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.635514 17890 consensus_queue.cc:237] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.625777 17928 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.664145 17870 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:09.686836 17870 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.688839 17870 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.667562 17975 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.690361 11293 catalog_manager.cc:5697] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.689345 11399 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:18:09.713568 17957 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.714133 17957 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.716257 17957 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.717187 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:09.717658 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 50021.5us
I20260709 13:18:09.718003 11468 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
W20260709 13:18:09.719056 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 43589us
I20260709 13:18:09.719668 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:09.720530 17909 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:09.720907 17909 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.721271 17909 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.721779 11294 catalog_manager.cc:5697] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.723088 17975 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.723387 11294 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 8 to 9, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 9 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.725231 17975 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.725713 17975 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:09.726219 17975 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.729792 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.730398 12112 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:09.730440 17824 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.731273 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:09.731094 17824 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.733935 17824 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.732201 17984 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:09.734498 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.734512 17984 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:09.735270 17984 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.734022 17974 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.737493 17974 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.744706 17984 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.745976 17870 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.732194 12122 raft_consensus.cc:1275] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:09.747151 17984 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.747417 17870 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.749421 17870 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.752755 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:09.753744 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.754565 11468 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
W20260709 13:18:09.756991 17989 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 13:18:09.761999 11441 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.762482 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.763167 11477 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:18:09.764000 17982 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:09.764369 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.764678 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.764467 17982 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.765197 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.765309 17982 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:09.766079 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.766659 11472 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:18:09.765267 12117 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:18:09.769412 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.770625 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.771384 12122 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:09.711094 11293 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.772311 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:09.777879 11468 raft_consensus.cc:1275] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:18:09.780545 12122 raft_consensus.cc:1275] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:09.783854 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.784615 12122 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:09.783478 11441 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.797197 11515 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:09.788820 17896 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:09.800712 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.801390 11397 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:18:09.801146 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.788245 17853 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:18:09.808369 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.829671 11515 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:18:09.831740 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 79064.3us
I20260709 13:18:09.835177 11471 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 12.
I20260709 13:18:09.839614 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.840611 12122 raft_consensus.cc:2393] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 21: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:09.800565 17896 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.839921 11514 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:09.846115 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.846901 12123 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:09.849385 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.850075 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.850651 12119 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:18:09.826443 17962 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:09.794862 17835 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:18:09.854665 12117 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:18:09.858383 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 193235us
I20260709 13:18:09.840971 17894 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:09.838063 17969 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:09.848644 17896 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.846400 17984 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:09.881038 17984 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.881475 17984 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:09.881270 17982 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.862993 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:09.883416 12112 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 12: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:09.863300 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.847821 17961 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:09.884240 17961 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.884635 17961 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.862054 17894 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.885530 17894 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:09.884249 11550 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:09.870548 17987 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:09.871228 17992 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.889300 17992 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:09.871634 17910 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:18:09.871932 17983 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:09.893342 17983 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.893747 17983 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.841193 17909 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.853534 17990 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:09.897090 17990 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.855521 17962 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.899892 17962 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:09.900774 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.902498 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.903119 11546 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.905714 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.906327 11469 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:09.906648 17909 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.875198 17856 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:09.877799 17971 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:09.907548 17971 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:09.907817 11472 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.908449 17971 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.911588 17992 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.911772 11476 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:18:09.850975 17870 consensus_queue.cc:1048] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:18:09.912396 17990 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.911080 17984 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.917847 11393 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:09.918231 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.919004 11397 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 16: Already voted for candidate 147201d22a694ca59f6e014f083a0892 in this term.
I20260709 13:18:09.919404 11400 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:18:09.919716 17961 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.920228 11472 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:18:09.920914 12123 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:09.922163 11402 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:09.922593 11546 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:18:09.914903 17896 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.861883 17851 consensus_queue.cc:1048] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:18:09.925231 17983 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.927399 17983 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.928819 17962 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:09.930408 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.931236 11546 raft_consensus.cc:2368] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:18:09.868430 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.931674 11549 raft_consensus.cc:1275] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:18:09.932188 11542 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:09.934465 17896 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:09.934828 17962 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:09.946374 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:09.942247 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.947180 11471 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:09.939298 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:09.948135 11395 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:09.939476 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:09.949273 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:09.950179 11550 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:09.951910 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:09.952535 12116 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
W20260709 13:18:09.953673 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 94293.4us
I20260709 13:18:09.947371 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:09.957785 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 75870.9us
I20260709 13:18:09.958568 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.964811 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.965715 11550 raft_consensus.cc:2368] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 21. Current term is 22.
I20260709 13:18:09.967335 11398 raft_consensus.cc:2368] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
I20260709 13:18:09.970834 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.973201 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:09.973999 11542 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:09.976059 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.971971 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.977684 11441 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.965588 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:09.908483 17856 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:09.979908 11402 raft_consensus.cc:3060] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:09.979977 11548 raft_consensus.cc:2368] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:18:09.983855 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:09.986245 11471 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:09.972307 17969 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:09.988057 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:09.955237 17890 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.991474 11550 raft_consensus.cc:2368] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:09.966279 17870 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:09.998567 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:10.002344 12115 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:09.999998 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.002857 11469 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.993716 12083 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:09.997846 12085 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:10.003352 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:10.018371 11395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:10.020007 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:10.020674 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:09.989297 17905 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:09.997177 17969 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.995568 17909 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:10.025354 11442 leader_election.cc:400] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:18:10.027511 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.029064 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:10.031260 12114 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 13: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:10.025427 17905 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:10.033986 17905 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.035631 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:10.036836 17998 log.cc:927] Time spent T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 13:18:10.022356 17870 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:10.038007 17870 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.043640 12085 leader_election.cc:400] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
I20260709 13:18:10.044013 12085 leader_election.cc:403] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.056196 11402 raft_consensus.cc:2468] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:10.062397 17882 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:18:10.065160 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:10.024572 17974 raft_consensus.cc:2804] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:10.032594 17969 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:10.044603 17930 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:10.033277 17909 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:10.044420 17982 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.015489 17890 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.020478 17878 consensus_queue.cc:1048] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:18:10.057449 17896 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:10.098084 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69121.5us
I20260709 13:18:10.078320 17824 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:18:10.096725 17909 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:10.130692 11398 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.085433 17974 raft_consensus.cc:697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:18:10.135365 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 57076.3us
I20260709 13:18:10.110505 17896 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.110738 17930 raft_consensus.cc:697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
W20260709 13:18:10.144858 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 98254.8us
I20260709 13:18:10.042220 11476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08ebf4312af74188830a07342d0dc119"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:10.148397 11476 raft_consensus.cc:606] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:18:10.153043 11367 leader_election.cc:400] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 21. Current term is 22.
I20260709 13:18:10.153402 11367 leader_election.cc:403] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.148556 11365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.123508 17975 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.156713 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:10.160526 11543 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:10.149415 17896 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:10.158283 17961 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:10.174005 17991 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:10.203925 18002 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:18:10.187846 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.208060 11398 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:10.213140 11469 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
W20260709 13:18:10.227859 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 79861.2us
I20260709 13:18:10.199959 17991 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.236671 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:10.249595 18010 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:18:10.257081 17969 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.258613 11549 raft_consensus.cc:2368] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16.
I20260709 13:18:10.234187 17890 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.268077 17882 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459) with higher term. Message: Invalid argument: T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15.
I20260709 13:18:10.268476 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:10.220032 17930 consensus_queue.cc:237] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.269084 11469 raft_consensus.cc:606] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:10.199086 17974 consensus_queue.cc:237] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.240296 17991 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:10.275238 17894 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.276654 17969 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:10.277549 17894 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:10.241881 18000 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:10.279071 11394 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:10.280974 11402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:10.283258 11397 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:10.284018 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:10.284559 11469 raft_consensus.cc:606] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Received request to transfer leadership to TS 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:10.206962 17905 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.289211 17905 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.290071 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:10.290716 11469 raft_consensus.cc:606] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:18:10.293286 17961 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.293416 11395 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:18:10.293907 17961 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.295223 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:10.296064 11469 raft_consensus.cc:606] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:18:10.296630 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.297382 11476 raft_consensus.cc:2393] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:10.303879 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:18:10.325146 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 176655us
I20260709 13:18:10.327220 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:10.328016 11476 raft_consensus.cc:2393] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 17: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:18:10.328130 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 97520.5us
I20260709 13:18:10.332068 11469 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:18:10.332643 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:10.335753 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.336410 11476 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.342909 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.344197 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:10.345495 11550 raft_consensus.cc:1275] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:18:10.345978 12112 raft_consensus.cc:1275] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:10.347941 12114 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:10.342970 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:10.350821 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.356245 18026 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:10.357697 17991 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.359917 18000 consensus_queue.cc:1048] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:10.362413 17961 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:10.363940 17837 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:10.369406 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:10.370146 11547 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:10.374737 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:10.375447 11471 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:10.372939 12085 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
W20260709 13:18:10.385654 18006 log.cc:927] Time spent T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.332s	user 0.005s	sys 0.000s
I20260709 13:18:10.385352 18010 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for earlier term 21. Current term is 22.
I20260709 13:18:10.399293 17995 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:10.400162 17958 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:10.400669 17958 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.400337 17896 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.401180 17958 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:10.403303 17896 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.409529 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:10.422506 18004 log.cc:927] Time spent T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.377s	user 0.000s	sys 0.003s
I20260709 13:18:10.437223 17991 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.439168 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.442219 11366 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.444015 17991 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:18:10.447166 18003 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.395s	user 0.003s	sys 0.000s
I20260709 13:18:10.456633 11398 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:10.398408 17909 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.453982 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:10.457890 17974 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:10.459815 11514 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.468519 17896 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:10.472051 17896 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:10.473001 17896 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:10.482510 18007 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:18:10.484177 18013 raft_consensus.cc:993] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:10.484676 18013 raft_consensus.cc:1081] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
W20260709 13:18:10.486320 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 109768us
I20260709 13:18:10.486575 17978 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:10.488932 17978 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:10.491757 12118 raft_consensus.cc:1275] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:10.494000 17976 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:10.494441 17976 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:10.495270 17976 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.496280 17976 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.496958 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.497534 12115 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:10.501459 18013 consensus_queue.cc:1048] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:10.507381 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:10.507243 17969 consensus_queue.cc:1048] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:10.510334 12112 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:18:10.515983 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 51034.3us
I20260709 13:18:10.507282 11296 catalog_manager.cc:5697] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 12 to 14, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:10.538709 11476 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:10.543624 12115 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:18:10.548405 11549 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:10.545382 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:10.551239 11398 consensus_queue.cc:237] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.548122 11365 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:10.564787 11400 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 22.
I20260709 13:18:10.573324 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.538266 17909 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.578682 17743 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:18:10.590075 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:10.590807 17868 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:10.592176 17851 raft_consensus.cc:993] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:10.592777 17851 raft_consensus.cc:1081] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:10.590713 12115 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:10.600003 17963 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:10.564548 17978 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:10.602131 17978 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:10.603021 17978 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.604167 17959 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:18:10.606110 17979 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:10.606978 11516 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:10.564636 17991 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:10.609836 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 52462.8us
I20260709 13:18:10.615999 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:10.617966 17824 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:10.585815 17984 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 22 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:10.628860 12115 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 22.
I20260709 13:18:10.618834 17991 raft_consensus.cc:697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:10.695338 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:10.645653 17824 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:10.705453 11471 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:10.711694 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.717638 11394 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:18:10.688812 11393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:18:10.690126 11400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:18:10.726076 11393 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:10.726598 11393 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.728005 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.715567 18009 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:10.754441 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.761065 11475 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:10.761096 11471 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:10.799119 11442 leader_election.cc:400] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
I20260709 13:18:10.799515 11442 leader_election.cc:403] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.804672 11476 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:10.805323 11442 leader_election.cc:400] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:10.805657 11442 leader_election.cc:403] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.815243 11442 leader_election.cc:400] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16.
I20260709 13:18:10.796746 18009 raft_consensus.cc:697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:10.822898 11397 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:10.806810 17856 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13.
W20260709 13:18:10.823817 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.810571 17987 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781) with higher term. Message: Invalid argument: T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:10.827440 11393 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.883035 18009 consensus_queue.cc:237] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:10.884649 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 67537.4us
I20260709 13:18:10.775655 17991 consensus_queue.cc:237] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.890204 11393 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:10.879907 17910 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:10.891225 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:10.895181 11471 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:10.895824 11471 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:10.812139 17985 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.250s
W20260709 13:18:10.881047 18030 log.cc:927] Time spent T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:18:10.895067 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:10.907801 11551 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.803396 17824 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.914343 11471 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.833262 17882 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:18:10.931849 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:10.939541 11514 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:10.942246 11471 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:10.949904 17959 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:10.966056 11471 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:10.967634 11365 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:10.968750 17965 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:10.956624 17971 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:10.969365 17965 raft_consensus.cc:697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:10.976279 17971 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
W20260709 13:18:10.976706 18034 log.cc:927] Time spent T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.225s	user 0.003s	sys 0.000s
I20260709 13:18:10.977066 17965 consensus_queue.cc:237] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:10.977280 17971 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:10.982909 18043 log.cc:927] Time spent T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:18:10.983819 18039 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:18:10.985630 18040 log.cc:927] Time spent T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:18:10.987808 18047 log.cc:927] Time spent T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:18:10.944517 17835 consensus_queue.cc:1048] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:18:10.999387 11400 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:10.999830 11400 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:11.006559 11475 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 14.
I20260709 13:18:11.020851 11400 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:10.960153 17984 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:18:11.036442 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:11.038527 11475 raft_consensus.cc:3055] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:11.038803 11400 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:11.038852 11475 raft_consensus.cc:740] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:11.039887 11475 consensus_queue.cc:260] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.036780 17909 raft_consensus.cc:993] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:11.081280 17743 raft_consensus.cc:1081] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:11.095012 11397 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 16. Current term is 17. Ops: []
W20260709 13:18:11.101298 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 168243us
I20260709 13:18:11.102622 17743 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:11.103435 17743 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:11.103818 17743 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:11.104727 17743 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.114665 17743 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:11.138754 11400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08ebf4312af74188830a07342d0dc119"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:18:11.140167 11400 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:11.140658 11400 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:11.144162 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 190670us
W20260709 13:18:11.160985 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 103290us
I20260709 13:18:11.165391 12118 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:11.171082 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:11.171984 12119 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:11.179962 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:11.186112 12113 raft_consensus.cc:2368] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:11.188156 11400 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.199112 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:11.199950 11471 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:11.200318 11471 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:11.211993 11551 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:11.175689 18010 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.218619 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47649.1us
I20260709 13:18:11.220773 11400 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:11.224063 11471 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.238294 11471 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:11.232733 11469 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:11.234004 11476 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:11.240875 11441 leader_election.cc:400] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:11.241282 11441 leader_election.cc:403] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.244223 17743 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993) with higher term. Message: Invalid argument: T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for earlier term 13. Current term is 14.
I20260709 13:18:11.265386 11396 raft_consensus.cc:1275] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:11.271713 11471 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:18:11.272966 11365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
W20260709 13:18:11.304699 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 156009us
I20260709 13:18:11.247946 18010 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.329968 11468 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:18:11.272828 17995 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 13:18:11.336267 18059 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:18:11.336326 18051 log.cc:927] Time spent T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:18:11.336938 18057 log.cc:927] Time spent T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:18:11.331105 18049 log.cc:927] Time spent T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:18:11.304227 18056 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:18:11.336146 18001 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:11.350416 18010 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:11.350775 18001 raft_consensus.cc:697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:11.351873 18001 consensus_queue.cc:237] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:11.355681 17958 consensus_meta.cc:287] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: Time spent flushing consensus metadata: real 0.954s	user 0.013s	sys 0.000s
I20260709 13:18:11.355450 18063 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:11.356164 17958 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.358211 17958 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:11.363689 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:11.364890 12119 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:11.372081 11475 raft_consensus.cc:3060] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:11.378854 11366 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:11.386337 11548 raft_consensus.cc:1275] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:11.383481 18010 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:18:11.387812 17965 raft_consensus.cc:2804] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:11.388486 17965 raft_consensus.cc:697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:11.389426 17965 consensus_queue.cc:237] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:11.390196 18042 log.cc:927] Time spent T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.496s	user 0.000s	sys 0.002s
I20260709 13:18:11.391618 17856 raft_consensus.cc:993] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:18:11.392086 17856 raft_consensus.cc:1081] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:18:11.393066 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:11.393700 12119 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:11.395860 11294 catalog_manager.cc:5697] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 17 to 19, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 19 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:11.402999 11475 raft_consensus.cc:2468] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:11.410800 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:11.435079 12119 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:18:11.439368 11472 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:11.430480 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:11.442883 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 202182us
I20260709 13:18:11.422364 17834 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:11.506340 17974 consensus_queue.cc:1048] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:18:11.521458 17834 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:11.511785 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 56889.9us
I20260709 13:18:11.524201 17834 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:11.527989 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.519098 11548 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:18:11.532785 11548 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:11.533164 11548 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:11.534649 18078 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:18:11.536149 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:11.537562 11443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.539454 11472 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:18:11.540527 11366 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:11.541383 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:11.546605 11366 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:11.547878 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:11.447196 17959 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:18:11.561685 17991 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:11.560508 17834 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:18:11.589231 11548 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.589980 18053 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:11.583957 18056 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:11.596570 18056 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.596069 11476 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:11.601871 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:11.602319 11476 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:11.602511 11476 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:11.605036 18053 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:11.608757 18067 log.cc:927] Time spent T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.290s	user 0.008s	sys 0.002s
I20260709 13:18:11.610387 11548 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:11.617833 11549 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:11.617416 11476 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.627426 11476 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:11.703666 11476 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:18:11.725822 11296 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 393) took 1559 ms. Trace:
W20260709 13:18:11.735692 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.735755 17962 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:11.737349 17962 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.746186 17963 raft_consensus.cc:993] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 9e9c2603c0bd4d25b713649b3890c46b to start an election
I20260709 13:18:11.746922 17963 raft_consensus.cc:1081] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Signalling peer 9e9c2603c0bd4d25b713649b3890c46b to start an election
W20260709 13:18:11.778568 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45307.6us
I20260709 13:18:11.780033 18052 consensus_queue.cc:1048] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:11.726933 11296 rpcz_store.cc:276] 0709 13:18:10.166805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.200814 (+ 34009us) service_pool.cc:224] Handling call
0709 13:18:11.725693 (+1524879us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:10.663549 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:10.664155 (+   606us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:10.664199 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:18:10.664223 (+    24us) tablet.cc:662] Decoding operations
0709 13:18:10.664595 (+   372us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:10.664673 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:10.664708 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:10.664911 (+   203us) tablet.cc:701] Row locks acquired
0709 13:18:10.664931 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:18:10.665081 (+   150us) write_op.cc:270] Start()
0709 13:18:10.694793 (+ 29712us) write_op.cc:276] Timestamp: P: 1783603090665026 usec, L: 0
0709 13:18:10.714732 (+ 19939us) op_driver.cc:348] REPLICATION: starting
0709 13:18:10.794069 (+ 79337us) log.cc:844] Serialized 527 byte log entry
0709 13:18:11.263209 (+469140us) op_driver.cc:464] REPLICATION: finished
0709 13:18:11.544281 (+281072us) write_op.cc:301] APPLY: starting
0709 13:18:11.549159 (+  4878us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:11.614183 (+ 65024us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:11.624104 (+  9921us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:11.624961 (+   857us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:11.625167 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:18:11.625218 (+    51us) write_op.cc:312] APPLY: finished
0709 13:18:11.625401 (+   183us) log.cc:844] Serialized 25 byte log entry
0709 13:18:11.630599 (+  5198us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:11.632037 (+  1438us) write_op.cc:454] Released schema lock
0709 13:18:11.641448 (+  9411us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:11.654887 (+ 13439us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:11.654915 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":301922,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":170823,"apply.run_cpu_time_us":4650,"apply.run_wall_time_us":151540,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":92372,"prepare.run_cpu_time_us":9976,"prepare.run_wall_time_us":142534,"replication_time_us":522742,"spinlock_wait_cycles":1508096,"thread_start_us":201871,"threads_started":2}]]}
I20260709 13:18:11.836858 18055 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:11.837421 18092 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:18:11.837631 18055 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.839075 12113 raft_consensus.cc:1240] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:11.843847 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:11.844436 11471 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:11.851514 11471 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:11.862722 11398 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:11.847749 17962 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:11.871552 11475 raft_consensus.cc:1275] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:11.872092 17976 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.872962 11473 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:11.871904 12117 raft_consensus.cc:1275] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:11.873413 17976 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:11.875888 17976 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:11.879208 17976 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:11.880460 11397 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:18:11.879943 17976 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:11.895872 18010 consensus_queue.cc:1048] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:11.938467 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.861949 17965 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:11.987367 12117 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:18:11.988996 18055 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:11.942415 18001 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:11.992498 18056 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:11.994307 17976 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:11.943149 17965 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.004818 17965 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:12.013167 17969 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:12.015105 17930 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.015717 17930 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.017822 11401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.018883 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.022020 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.018350 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.033191 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:12.042752 11468 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:18:12.052317 12117 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:12.063410 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.069718 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:12.070353 12113 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:12.071527 12114 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:12.078447 12113 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 13.
I20260709 13:18:12.114399 11294 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 409) took 1558 ms. Trace:
I20260709 13:18:12.109160 11551 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:12.105160 11514 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:12.111995 11550 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
 from {username='slave'} at 127.0.0.1:43442
I20260709 13:18:12.131764 11550 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:12.085068 11549 raft_consensus.cc:1275] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:18:12.132314 11550 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:12.132822 11549 pending_rounds.cc:85] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 10
I20260709 13:18:12.112257 12117 raft_consensus.cc:1275] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:18:12.102705 11547 raft_consensus.cc:1275] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:12.083086 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:12.114837 11298 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:12.147385 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.148921 18056 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:12.149040 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.149542 18056 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:12.150663 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45578.1us
W20260709 13:18:12.150925 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 85411.3us
I20260709 13:18:12.151185 17976 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:12.151713 17976 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:12.133190 11549 pending_rounds.cc:107] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:18:12.152696 11549 raft_consensus.cc:2889] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:12.152563 17976 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.154619 11298 catalog_manager.cc:5697] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:12.163286 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.162205 11298 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 18 to 19, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 19 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:12.165689 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.168068 11298 catalog_manager.cc:5697] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:12.153558 18024 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:18:12.184662 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:12.165479 12083 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:12.231113 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65753.9us
I20260709 13:18:12.243074 17958 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:12.242369 17951 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.247947 17951 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:12.248339 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 72130.7us
I20260709 13:18:12.250162 17951 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:12.251880 18102 consensus_queue.cc:1048] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:12.252804 17958 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:12.185196 17930 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:12.175004 17959 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:18:12.171936 18012 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:18:12.347177 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.185680 17971 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:18:12.307811 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:12.256870 18069 consensus_queue.cc:1048] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 13:18:12.349943 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 96162.6us
I20260709 13:18:12.129640 11294 rpcz_store.cc:276] 0709 13:18:10.555505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.555652 (+   147us) service_pool.cc:224] Handling call
0709 13:18:12.114314 (+1558662us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:11.596067 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:11.596587 (+   520us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:11.596622 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:18:11.596639 (+    17us) tablet.cc:662] Decoding operations
0709 13:18:11.596957 (+   318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:11.597005 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:11.597031 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:11.597228 (+   197us) tablet.cc:701] Row locks acquired
0709 13:18:11.597245 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:18:11.597382 (+   137us) write_op.cc:270] Start()
0709 13:18:11.597492 (+   110us) write_op.cc:276] Timestamp: P: 1783603091597335 usec, L: 0
0709 13:18:11.597518 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:18:11.607246 (+  9728us) log.cc:844] Serialized 532 byte log entry
0709 13:18:11.622836 (+ 15590us) op_driver.cc:464] REPLICATION: finished
0709 13:18:11.651087 (+ 28251us) write_op.cc:301] APPLY: starting
0709 13:18:11.687122 (+ 36035us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:11.707221 (+ 20099us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:11.721681 (+ 14460us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:11.722886 (+  1205us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:11.742878 (+ 19992us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:18:11.742951 (+    73us) write_op.cc:312] APPLY: finished
0709 13:18:11.743652 (+   701us) log.cc:844] Serialized 25 byte log entry
0709 13:18:11.751244 (+  7592us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:11.751333 (+    89us) write_op.cc:454] Released schema lock
0709 13:18:11.762229 (+ 10896us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:11.784480 (+ 22251us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:11.784513 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":24655,"apply.run_cpu_time_us":4704,"apply.run_wall_time_us":133431,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":171009,"prepare.run_cpu_time_us":2478,"prepare.run_wall_time_us":16188,"replication_time_us":18362,"thread_start_us":169431,"threads_started":2}]]}
W20260709 13:18:12.377293 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 116774us
I20260709 13:18:12.356413 11473 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:12.356750 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 127425us
I20260709 13:18:12.354682 11550 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.413836 11397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.414155 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:12.419543 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:12.418401 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.441505 18010 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:12.416733 17963 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.418286 18024 consensus_queue.cc:1048] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:18:12.443624 18019 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:12.448294 11550 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:12.410912 18056 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:12.515908 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:12.439471 17958 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.526239 11475 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:12.526722 11475 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:12.527700 11475 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.529083 11475 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:12.532210 18100 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.298s	user 0.004s	sys 0.009s
W20260709 13:18:12.536643 18106 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:12.456120 17963 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.549664 17963 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:12.562243 11514 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:12.563890 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 382833us
I20260709 13:18:12.571043 11514 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:12.602599 17851 consensus_queue.cc:1059] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:12.574882 17969 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:18:12.601002 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:12.605077 11396 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 20.
I20260709 13:18:12.607362 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:12.573057 17975 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:12.630074 11475 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:12.619773 18050 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:18:12.633098 11401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.004s	sys 0.008s
W20260709 13:18:12.634119 11401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.005s	sys 0.009s
I20260709 13:18:12.621325 18069 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:12.628388 17975 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:12.633934 18050 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:12.658418 17910 log.cc:927] Time spent T 00000000000000000000000000000000 P d085878dd61a4a3984dff72a53c92b4d: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:12.658849 18050 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:12.684054 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:12.651141 18069 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:12.692163 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 300060us
I20260709 13:18:12.695206 12083 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:12.716533 12083 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:12.718924 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:12.683394 18052 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.724083 12083 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:12.728461 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:12.729211 12117 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:12.719090 17962 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:12.727926 18038 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:12.734927 17984 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.022s	sys 0.029s
I20260709 13:18:12.729048 17930 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:12.729393 17974 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:12.752808 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 150687us
W20260709 13:18:12.751626 17984 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.022s	sys 0.029s
I20260709 13:18:12.734836 18052 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.787122 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:12.815626 11550 raft_consensus.cc:2393] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 16: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:18:12.800839 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.784080 11471 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:18:12.789346 18108 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:18:12.927301 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 317039us
I20260709 13:18:13.012328 17984 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:13.017015 18129 consensus_queue.cc:1048] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:12.771539 11514 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:13.019099 17975 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:13.020125 17975 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:13.021487 17975 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.969441 17958 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:12.929250 18118 log.cc:927] Time spent T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:18:12.976068 18052 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:12.999085 17976 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.009s	sys 0.014s
W20260709 13:18:13.027339 17976 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.009s	sys 0.014s
I20260709 13:18:12.992514 18050 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.028263 17958 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.029853 18050 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 21 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:12.994946 18010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41722 (request call id 114) took 2485 ms. Trace:
I20260709 13:18:13.031339 17853 raft_consensus.cc:3055] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:13.031843 17853 raft_consensus.cc:740] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:13.030520 18010 rpcz_store.cc:276] 0709 13:18:10.509616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.528286 (+ 18670us) service_pool.cc:224] Handling call
0709 13:18:10.716649 (+188363us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4c002a08c0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da7be5a20 00007f8da7bf6a29 00007f8da7bbc644 00007f8da7be3f2d 00007f8da7be3b93 00007f8da98721af 00007f8da9871f3e 00007f8da98f7888 00007f8da991475c 00007f8da991576a 00007f8da991491b
0709 13:18:10.724122 (+  7473us) log.cc:844] Serialized 480 byte log entry
0709 13:18:10.973609 (+249487us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b5400185220. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9892142 00007f8da9872eaa 00007f8da9879d53 00007f8da9879cd9 00007f8da9879c69 00007f8da9879c31 00007f8da9878dfd 000000000038f354 000000000038f199 00007f8da682c475
0709 13:18:11.382480 (+408871us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b180042d1e0. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8daf6f4de1 00007f8daf7015b1 00007f8da67fa2a6 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9899b26 00007f8da989349a 00007f8da9892714
0709 13:18:11.750105 (+367625us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b5400185220. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9882571 00007f8da9899b26 00007f8da989349a 00007f8da9892714 00007f8da9872eaa 00007f8da9879d53 00007f8da9879cd9 00007f8da9879c69 00007f8da9879c31 00007f8da9878dfd 000000000038f354
0709 13:18:12.994801 (+1244696us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9986,"raft.queue_time_us":6077,"raft.run_cpu_time_us":28379,"raft.run_wall_time_us":1056686,"spinlock_wait_cycles":57755904,"thread_start_us":101896,"threads_started":1,"wal-append.queue_time_us":165598,"wal-append.run_cpu_time_us":1909,"wal-append.run_wall_time_us":1059000}
I20260709 13:18:13.032845 17853 consensus_queue.cc:260] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:13.034111 17853 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:13.035617 12119 raft_consensus.cc:1275] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:18:13.044332 11286 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c4eb2d16aa247998d0927eb34dedfa8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:13.053489 18115 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:13.053953 18115 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:13.054788 18115 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.135998 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:12.990247 18122 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.158675 18122 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:12.980527 18069 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:13.009429 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:13.173341 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:13.180526 11468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43630 (request call id 4429) took 1133 ms. Trace:
I20260709 13:18:13.181105 11468 rpcz_store.cc:276] 0709 13:18:12.046822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.047408 (+   586us) service_pool.cc:224] Handling call
0709 13:18:12.047690 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.048947 (+  1257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.048975 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.050880 (+  1905us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.466572 (+415692us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:12.466630 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.466726 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.180230 (+713504us) raft_consensus.cc:1685] finished
0709 13:18:13.180260 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.180414 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":395225,"threads_started":1,"wal-append.queue_time_us":580709}
W20260709 13:18:13.187014 11514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 415541us
W20260709 13:18:13.232673 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 214453us
I20260709 13:18:13.235149 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:13.239493 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:13.240100 12114 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:18:13.051154 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:13.246078 11399 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:13.253322 11399 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 21.
I20260709 13:18:13.254819 18064 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:13.255812 11294 catalog_manager.cc:5697] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 15 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:13.258832 11298 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38378 (request call id 271) took 2348 ms. Trace:
I20260709 13:18:13.264273 17976 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:18:13.283083 11473 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.002s	sys 0.010s
W20260709 13:18:13.291725 11473 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.002s	sys 0.010s
I20260709 13:18:13.284543 11294 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 14 to 15, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 15 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:13.283561 12115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.016s	sys 0.019s
W20260709 13:18:13.296686 12115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.016s	sys 0.019s
W20260709 13:18:13.266019 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 203395us
I20260709 13:18:13.268719 11543 raft_consensus.cc:1240] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:13.269740 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:13.038175 12122 raft_consensus.cc:2955] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:13.265000 17958 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:13.297777 12114 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 21.
I20260709 13:18:13.316907 11441 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:13.322618 11471 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:18:13.304401 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 67642us
I20260709 13:18:13.306733 12112 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:13.325991 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:13.277209 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:13.327014 18124 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:13.307758 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:13.285609 17976 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:13.330358 17976 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:13.333711 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:13.343809 12112 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:13.289966 18115 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:13.360713 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:13.375715 11544 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:13.341820 18013 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:13.285197 11298 rpcz_store.cc:276] 0709 13:18:10.910808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.911027 (+   219us) service_pool.cc:224] Handling call
0709 13:18:13.258749 (+2347722us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:12.603148 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:12.603890 (+   742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:12.603941 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:18:12.603972 (+    31us) tablet.cc:662] Decoding operations
0709 13:18:12.604693 (+   721us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:12.604761 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:12.604789 (+    28us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:12.605285 (+   496us) tablet.cc:701] Row locks acquired
0709 13:18:12.605311 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:18:12.605489 (+   178us) write_op.cc:270] Start()
0709 13:18:12.605620 (+   131us) write_op.cc:276] Timestamp: P: 1783603092605435 usec, L: 0
0709 13:18:12.605647 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:18:12.606460 (+   813us) log.cc:844] Serialized 1650 byte log entry
0709 13:18:12.970536 (+364076us) op_driver.cc:464] REPLICATION: finished
0709 13:18:13.010951 (+ 40415us) write_op.cc:301] APPLY: starting
0709 13:18:13.011078 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:13.013863 (+  2785us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:13.013964 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:13.016520 (+  2556us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:13.016722 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:18:13.016763 (+    41us) write_op.cc:312] APPLY: finished
0709 13:18:13.016976 (+   213us) log.cc:844] Serialized 49 byte log entry
0709 13:18:13.017138 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:13.017229 (+    91us) write_op.cc:454] Released schema lock
0709 13:18:13.017267 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:13.024296 (+  7029us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:13.024323 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":324997,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":40173,"apply.run_cpu_time_us":6456,"apply.run_wall_time_us":13282,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":391003,"prepare.run_cpu_time_us":3808,"prepare.run_wall_time_us":12565,"replication_time_us":364712,"spinlock_wait_cycles":9472,"thread_start_us":321865,"threads_started":2}]]}
I20260709 13:18:13.324301 17958 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:18:13.347043 17878 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:13.431980 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.456197 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.283522 18064 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:13.376844 18121 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:13.464906 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 120566us
I20260709 13:18:13.469645 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:13.467048 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 135783us
W20260709 13:18:13.472761 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 120165us
W20260709 13:18:13.471458 18148 log.cc:927] Time spent T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:18:13.468426 18001 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:13.488265 18001 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.496480 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:13.507480 18010 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:13.508209 18010 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.511716 18122 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:13.515219 18064 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:13.518517 18140 log.cc:927] Time spent T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:18:13.519397 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:13.532507 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 65109.7us
W20260709 13:18:13.538900 18142 log.cc:927] Time spent T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:18:13.471338 18121 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.548642 17878 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:13.490899 18001 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:13.555410 18065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.011s	sys 0.007s
W20260709 13:18:13.556020 18065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.008s
I20260709 13:18:13.557534 18010 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:13.572777 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 96217.2us
I20260709 13:18:13.581732 18155 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:13.612658 11400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:13.595667 18124 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:13.597747 18012 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.631361 12117 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:13.606467 18155 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.637411 18115 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.639462 18012 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:13.671857 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 277309us
I20260709 13:18:13.637645 18124 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:13.681668 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 308162us
I20260709 13:18:13.654811 17976 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:18:13.688804 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:13.661861 18115 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:13.673833 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:13.706513 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61793.5us
I20260709 13:18:13.681365 18019 consensus_queue.cc:1048] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:18:13.705402 18055 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:13.775537 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 52772.5us
W20260709 13:18:13.855903 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 165488us
W20260709 13:18:13.858068 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 132928us
I20260709 13:18:13.724195 18153 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:13.724794 18013 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:13.907943 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:13.911094 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:13.915654 11441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.916360 18050 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.919597 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:13.923309 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:13.916940 18050 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:13.949992 18169 log.cc:927] Time spent T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:18:13.953852 18012 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:13.955485 12085 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:13.959198 18155 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:13.959477 18162 log.cc:927] Time spent T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:18:13.959329 18038 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:13.963459 18168 log.cc:927] Time spent T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
I20260709 13:18:13.973237 18115 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:13.984314 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:13.990232 18166 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:13.999624 11440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.965390 18013 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:13.997094 18165 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:14.009078 18167 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:14.004343 18050 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.004017 18166 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.023943 18165 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:14.036347 18184 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:14.039764 18184 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.040611 18167 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.042143 18184 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.042949 18167 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.046723 18166 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.050596 12119 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 13:18:14.061697 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.062558 11396 raft_consensus.cc:2410] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:14.056233 18124 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:14.067032 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 59674.6us
I20260709 13:18:14.072204 11468 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:18:14.048555 18108 log.cc:927] Time spent T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.667s	user 0.005s	sys 0.003s
I20260709 13:18:14.080909 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.081975 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.083405 11473 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 13:18:14.084355 18165 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:14.106241 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.107939 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.108328 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.108770 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.109954 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.111685 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.112546 11468 raft_consensus.cc:2410] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:14.115175 18173 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:14.118399 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.126885 11549 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:18:14.127385 11549 pending_rounds.cc:85] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting all ops after (but not including) 14
I20260709 13:18:14.127671 11549 pending_rounds.cc:107] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:18:14.127897 11549 raft_consensus.cc:2889] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.143100 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
W20260709 13:18:14.224911 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 78244.5us
W20260709 13:18:14.230806 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 128197us
I20260709 13:18:14.236253 11514 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.237604 11514 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.237848 18170 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.237680 17976 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:18:14.238437 18170 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.239347 18130 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.239511 17976 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:14.240258 17976 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:14.240574 18170 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:14.241487 17979 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.241925 17979 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.242619 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.243822 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.243978 17979 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:14.239898 18130 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.147311 18173 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:14.253997 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.254017 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.231806 18172 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:18:14.262524 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.267705 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.299036 11514 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.299748 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.300995 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.304914 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.309670 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.309448 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.313333 17856 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:14.313913 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.315022 18180 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.287554 17979 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:18:14.315652 18180 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.251926 18159 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:18:14.318117 18180 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:14.265987 18156 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:14.269393 18181 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:14.319330 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.335753 17975 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:14.336761 17975 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:14.337419 17975 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.288009 18191 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:18:14.360050 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44539667cbc04657b801ff659dee5bde"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:14.360934 11473 raft_consensus.cc:606] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
W20260709 13:18:14.367189 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 56808.5us
W20260709 13:18:14.369503 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52916.4us
I20260709 13:18:14.326092 11544 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:14.379946 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.394556 18121 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:14.401566 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.407383 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:14.408046 11473 raft_consensus.cc:606] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.413882 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:14.414471 11473 raft_consensus.cc:606] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Received request to transfer leadership to TS 147201d22a694ca59f6e014f083a0892
I20260709 13:18:14.414794 11473 raft_consensus.cc:614] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:14.424644 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.428565 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:14.445348 18189 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:14.447259 11473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:56646
W20260709 13:18:14.449200 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 175722us
I20260709 13:18:14.449534 11473 raft_consensus.cc:606] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.445485 18183 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.450289 12083 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.461802 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.464316 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:14.509603 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 45425.1us
I20260709 13:18:14.451159 18189 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:14.512006 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.513806 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:14.526701 18190 log.cc:927] Time spent T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: Append to log took a long time: real 0.236s	user 0.000s	sys 0.005s
I20260709 13:18:14.472491 18052 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:14.471598 17969 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:14.527951 17975 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.528391 17969 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } attrs { replace: true } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.466658 18121 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:14.460248 18183 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:14.522147 18189 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:14.466223 18170 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.538585 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.542892 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.560403 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.561508 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.563045 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.574412 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.576359 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.578394 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.579063 12114 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 12.
I20260709 13:18:14.580142 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.588097 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.573251 18181 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:14.601166 18101 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.601629 18181 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.603941 18181 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:14.609771 18200 log.cc:927] Time spent T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:18:14.549320 18170 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.614308 18170 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:14.617570 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.618075 18183 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:14.619134 17969 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.620599 17958 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:14.623939 18130 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.626827 12083 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.601702 18101 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.600575 11516 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.644234 11441 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.645864 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.652225 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.654263 11515 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.655651 11441 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.667459 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.668447 12112 raft_consensus.cc:1813] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Handling vote request from an unknown peer 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:14.669260 12112 raft_consensus.cc:2410] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:18:14.672735 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.673106 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.674943 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.657680 18130 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:14.844075 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 162256us
I20260709 13:18:14.844879 18129 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.845598 18129 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.848090 18129 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.851825 18189 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.852952 11476 raft_consensus.cc:1240] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:14.860219 18128 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:14.860805 18128 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.840113 18174 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.866070 18063 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:14.866915 18174 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.872426 18191 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.872859 12083 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.872933 18191 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:14.880798 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 199006us
I20260709 13:18:14.883352 18191 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.883451 11471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:56646
I20260709 13:18:14.884325 11471 raft_consensus.cc:606] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.887118 18166 raft_consensus.cc:993] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:14.887616 18166 raft_consensus.cc:1081] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:14.892554 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.898677 11441 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:14.900255 18207 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:14.901352 11294 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34874 (request call id 366) took 2479 ms. Trace:
I20260709 13:18:14.901484 18055 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:14.904728 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:14.905906 18189 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.907168 18174 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:14.862773 18128 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.910143 12118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55074
W20260709 13:18:14.818922 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 228090us
I20260709 13:18:14.818307 18121 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:14.823192 18187 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:14.759111 18019 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:14.675880 17824 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:14.688505 18183 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:14.945734 18183 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:14.946178 18183 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.947837 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:14.823073 18161 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:14.916977 12118 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.952390 12118 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:14.919291 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.921162 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.922221 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:14.925748 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.939273 12085 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:14.922019 11293 catalog_manager.cc:5697] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.937557 12112 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:18:14.901770 11294 rpcz_store.cc:276] 0709 13:18:12.422411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.438519 (+ 16108us) service_pool.cc:224] Handling call
0709 13:18:12.774279 (+335760us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5400002490. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da66b4db1 00007f8da67f5640 00007f8daf6ccb76 00007f8daf7d6c3e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:18:14.901293 (+2127014us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:13.662495 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:13.697094 (+ 34599us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:13.700753 (+  3659us) write_op.cc:435] Acquired schema lock
0709 13:18:13.703071 (+  2318us) tablet.cc:662] Decoding operations
0709 13:18:13.866189 (+163118us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:13.909400 (+ 43211us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:13.911924 (+  2524us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:18:13.941984 (+ 30060us) tablet.cc:701] Row locks acquired
0709 13:18:13.944988 (+  3004us) write_op.cc:260] PREPARE: finished
0709 13:18:13.954943 (+  9955us) write_op.cc:270] Start()
0709 13:18:13.955086 (+   143us) write_op.cc:276] Timestamp: P: 1783603093954884 usec, L: 0
0709 13:18:13.955109 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:13.989008 (+ 33899us) log.cc:844] Serialized 906 byte log entry
0709 13:18:14.378362 (+389354us) op_driver.cc:464] REPLICATION: finished
0709 13:18:14.841184 (+462822us) write_op.cc:301] APPLY: starting
0709 13:18:14.841267 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:14.872878 (+ 31611us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:14.872964 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:14.874667 (+  1703us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:14.874869 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:18:14.874915 (+    46us) write_op.cc:312] APPLY: finished
0709 13:18:14.875125 (+   210us) log.cc:844] Serialized 33 byte log entry
0709 13:18:14.875279 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:14.875385 (+   106us) write_op.cc:454] Released schema lock
0709 13:18:14.875423 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:14.876277 (+   854us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:14.876305 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":114939,"reported_tablets":7,"spinlock_wait_cycles":52552064,"child_traces":[["op",{"apply.queue_time_us":421595,"apply.run_cpu_time_us":4676,"apply.run_wall_time_us":35083,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":267459,"prepare.run_cpu_time_us":4440,"prepare.run_wall_time_us":341286,"replication_time_us":356282,"thread_start_us":406882,"threads_started":2}]]}
I20260709 13:18:14.942574 17856 raft_consensus.cc:993] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:14.967893 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.971215 18052 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:14.972321 11293 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 17 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.970726 12118 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:14.974468 18101 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:14.973966 11293 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 11 to 13, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.975545 11293 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 11 to 12, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 12 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.976013 12118 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:14.976307 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 55509.8us
I20260709 13:18:14.977357 11293 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 12 to 14, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 14 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.979419 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.982509 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:14.986622 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:14.992004 17958 consensus_queue.cc:1059] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:14.986245 18207 raft_consensus.cc:1081] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:14.993841 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:14.994486 11396 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 21.
I20260709 13:18:15.002780 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:15.003468 11393 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:15.004675 11400 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:18:15.006711 18207 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:18:15.006978 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:15.007505 11472 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:15.007784 11472 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:15.008462 11472 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.009382 11472 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:14.987419 11543 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:14.966945 18019 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:15.019845 18019 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:15.029167 11442 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.034494 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:15.041445 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:18:15.043651 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 46874.1us
I20260709 13:18:15.043731 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:15.045521 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 56486.8us
I20260709 13:18:15.048540 11472 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:18:15.046417 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:15.051643 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:15.052016 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:15.052609 11543 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:15.065673 11543 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:18:15.066737 12116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55074
I20260709 13:18:15.068589 17856 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:15.038341 17868 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:15.071251 18207 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:15.071722 18207 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.072083 18207 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.080627 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:15.080964 11514 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.082244 11514 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.084069 17959 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.084240 17976 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.089953 12116 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:18:15.091696 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 47361us
I20260709 13:18:15.096066 12116 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.096333 11516 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.068632 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:15.091377 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:15.042551 17979 raft_consensus.cc:3055] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:15.102710 11365 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:15.112960 11366 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.101724 17979 raft_consensus.cc:740] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:15.110049 18055 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:15.116643 17969 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:15.132237 18173 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:15.122668 17878 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:15.150210 18183 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.164511 12116 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:15.168763 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 48977.6us
I20260709 13:18:15.172745 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:15.173583 11473 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:15.173980 11473 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:15.166883 12116 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:15.182583 11471 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:15.200006 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 76772.6us
I20260709 13:18:15.178921 11473 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.201929 11473 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:15.205361 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 84584.2us
I20260709 13:18:15.206965 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:15.207594 11543 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.181200 18055 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:15.227758 11543 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:15.229846 12085 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:15.163972 17979 consensus_queue.cc:260] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.214681 18121 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:15.245205 18055 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.240787 18156 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:15.249702 18156 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:15.250530 18156 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.268573 11473 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:15.292776 18183 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:15.295097 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:15.298354 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:15.302419 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:15.309211 18165 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:15.309301 18207 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:15.311697 18207 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:15.297001 18128 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:18:15.314082 17979 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:15.315553 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:15.315860 11398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:15.316227 11543 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.316443 11398 raft_consensus.cc:3060] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:15.314463 18128 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:15.324172 18128 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:15.325073 18128 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.328616 18128 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.337769 18121 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:15.338322 18121 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:15.353650 17856 raft_consensus.cc:993] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:15.354265 17856 raft_consensus.cc:1081] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Signalling peer 147201d22a694ca59f6e014f083a0892 to start an election
I20260709 13:18:15.361735 11398 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:15.382258 11543 raft_consensus.cc:2468] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:15.391191 11293 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34878 (request call id 394) took 3530 ms. Trace:
I20260709 13:18:15.392199 11298 catalog_manager.cc:5697] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 15 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:15.394305 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 65164.6us
I20260709 13:18:15.394420 11298 catalog_manager.cc:5697] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 16 to 17, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.400017 11399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44539667cbc04657b801ff659dee5bde"
dest_uuid: "147201d22a694ca59f6e014f083a0892"
 from {username='slave'} at 127.0.0.1:41728
I20260709 13:18:15.400589 11399 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.400967 11399 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.404254 11298 catalog_manager.cc:5697] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 13 to 14, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 14 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.406173 11298 catalog_manager.cc:5697] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 18 to 19, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 19 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.393334 11293 rpcz_store.cc:276] 0709 13:18:11.860344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.861175 (+   831us) service_pool.cc:224] Handling call
0709 13:18:15.391128 (+3529953us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:15.056652 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:15.201667 (+145015us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:15.204035 (+  2368us) write_op.cc:435] Acquired schema lock
0709 13:18:15.204060 (+    25us) tablet.cc:662] Decoding operations
0709 13:18:15.251809 (+ 47749us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:15.251864 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:15.251891 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:18:15.252410 (+   519us) tablet.cc:701] Row locks acquired
0709 13:18:15.252428 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:18:15.252565 (+   137us) write_op.cc:270] Start()
0709 13:18:15.252676 (+   111us) write_op.cc:276] Timestamp: P: 1783603095252517 usec, L: 0
0709 13:18:15.252698 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:18:15.253310 (+   612us) log.cc:844] Serialized 2020 byte log entry
0709 13:18:15.320060 (+ 66750us) op_driver.cc:464] REPLICATION: finished
0709 13:18:15.320379 (+   319us) write_op.cc:301] APPLY: starting
0709 13:18:15.320464 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:15.322672 (+  2208us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:15.322739 (+    67us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:15.325566 (+  2827us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:15.325756 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:18:15.325815 (+    59us) write_op.cc:312] APPLY: finished
0709 13:18:15.326049 (+   234us) log.cc:844] Serialized 57 byte log entry
0709 13:18:15.326222 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:15.326320 (+    98us) write_op.cc:454] Released schema lock
0709 13:18:15.343038 (+ 16718us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:15.344750 (+  1712us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:15.344782 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":271532,"reported_tablets":15,"spinlock_wait_cycles":21248,"child_traces":[["op",{"apply.queue_time_us":132,"apply.run_cpu_time_us":6345,"apply.run_wall_time_us":24312,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":49552,"prepare.run_cpu_time_us":4226,"prepare.run_wall_time_us":196946,"replication_time_us":67223,"thread_start_us":49226,"threads_started":1}]]}
I20260709 13:18:15.396950 11442 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:15.423454 17853 raft_consensus.cc:2804] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:15.424202 17853 raft_consensus.cc:697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:15.425072 17853 consensus_queue.cc:237] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:15.414559 11399 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.435793 11399 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:15.436777 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:15.438259 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:15.450021 11476 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:15.455322 12119 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.455370 11476 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:15.462100 18207 raft_consensus.cc:993] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:15.462646 18183 raft_consensus.cc:1081] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:15.456489 11476 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.464663 11476 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.466674 12116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55074
I20260709 13:18:15.467346 12116 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.467759 12116 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.446257 18121 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.473063 18121 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 22 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:15.473708 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:15.475950 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:15.478209 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:15.479149 17979 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:18:15.484539 12119 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:18:15.492671 11366 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:15.495900 11476 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:18:15.496483 18154 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:15.503629 18154 raft_consensus.cc:697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:15.504626 18154 consensus_queue.cc:237] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.536549 11296 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 15 to 16, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 16 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.538399 11296 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 17 to 18, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:15.540297 11296 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 12 to 13, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 13 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.559963 12116 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.585132 11298 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34858 (request call id 410) took 1110 ms. Trace:
I20260709 13:18:15.587118 12116 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:15.586196 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:15.589308 11544 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.589205 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:15.590040 11476 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:15.590440 11476 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:15.585606 11298 rpcz_store.cc:276] 0709 13:18:14.474387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.518024 (+ 43637us) service_pool.cc:224] Handling call
0709 13:18:14.876475 (+358451us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b54000181d4. stack: 00007f8da67fa343 00007f8da67fa240 00007f8da61891d4 00007f8dafec7304 00007f8dafec7299 00007f8da9883090 00007f8da9921f3a 00007f8daf6d81cc 00007f8daf7d637e 00007f8dadc972a4 00007f8dadc97232 00007f8dadc97161 00007f8dadc970dc 00007f8dadc96392 00007f8da7c5448c 00007f8da7c538c6
0709 13:18:15.585072 (+708597us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:15.449077 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:15.450328 (+  1251us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:15.450360 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:18:15.450372 (+    12us) tablet.cc:662] Decoding operations
0709 13:18:15.450932 (+   560us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:15.451009 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:15.451034 (+    25us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:15.451367 (+   333us) tablet.cc:701] Row locks acquired
0709 13:18:15.451383 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:18:15.451519 (+   136us) write_op.cc:270] Start()
0709 13:18:15.451631 (+   112us) write_op.cc:276] Timestamp: P: 1783603095451476 usec, L: 0
0709 13:18:15.451652 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:18:15.455474 (+  3822us) log.cc:844] Serialized 1660 byte log entry
0709 13:18:15.457394 (+  1920us) op_driver.cc:464] REPLICATION: finished
0709 13:18:15.457819 (+   425us) write_op.cc:301] APPLY: starting
0709 13:18:15.457893 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:15.460196 (+  2303us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:15.460283 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:15.462709 (+  2426us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:15.462871 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:18:15.462955 (+    84us) write_op.cc:312] APPLY: finished
0709 13:18:15.474492 (+ 11537us) log.cc:844] Serialized 49 byte log entry
0709 13:18:15.474656 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:15.474747 (+    91us) write_op.cc:454] Released schema lock
0709 13:18:15.474790 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:15.475125 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:15.475150 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":32763,"reported_tablets":14,"spinlock_wait_cycles":146255744,"child_traces":[["op",{"apply.queue_time_us":223,"apply.run_cpu_time_us":5824,"apply.run_wall_time_us":17280,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16728,"num_ops":4,"prepare.queue_time_us":2632,"prepare.run_cpu_time_us":2958,"prepare.run_wall_time_us":6718,"replication_time_us":5415}]]}
I20260709 13:18:15.595510 11476 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.603405 11476 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:15.610411 11476 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:15.612511 12083 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:15.613336 18121 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:15.613883 18121 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:15.614854 18121 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:15.610792 11544 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:15.639020 11294 catalog_manager.cc:5697] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: config changed from index 9 to 10, VOTER 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) evicted. New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:15.664644 11544 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:15.666697 17979 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:15.690091 11293 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.705832 11538 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c4eb2d16aa247998d0927eb34dedfa8 with delete_type TABLET_DATA_TOMBSTONED (TS 9e9c2603c0bd4d25b713649b3890c46b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51266
I20260709 13:18:15.716097 11298 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 8 to 9, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 9 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:15.725229 11476 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:15.732954 11544 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:15.730538 18055 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:15.743942 11296 catalog_manager.cc:5697] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 9 to 10, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 10 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.759063 17979 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:18:15.779366 18230 tablet_replica.cc:333] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:15.780448 18230 raft_consensus.cc:2243] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:15.783653 18230 raft_consensus.cc:2272] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:15.786568 18230 ts_tablet_manager.cc:1916] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:15.799208 11473 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:15.800822 18055 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:15.811026 11294 catalog_manager.cc:5697] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.882329 11399 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:15.884975 12116 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Rejecting Update request from peer 34fe1737ebb840fb9746980f649cc0a6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:18:15.885859 18165 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:15.893249 11293 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 17 to 18, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.908450 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 17 to 18, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.943799 12113 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:18:15.911499 18184 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 13:18:15.951354 18154 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:18:15.976969 11473 raft_consensus.cc:1275] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:18:15.979940 18185 consensus_queue.cc:1048] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:16.045272 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:16.063175 18183 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:18:16.063642 18183 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:16.034755 18230 ts_tablet_manager.cc:1929] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
I20260709 13:18:16.071389 18183 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.084822 18183 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:16.091817 17979 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:16.092454 17979 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.094383 17979 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:16.100952 18230 log.cc:1199] T 1c4eb2d16aa247998d0927eb34dedfa8 P 9e9c2603c0bd4d25b713649b3890c46b: Deleting WAL directory at /tmp/dist-test-taskvp1vQM/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602855680335-10252-0/minicluster-data/ts-2-root/wals/1c4eb2d16aa247998d0927eb34dedfa8
I20260709 13:18:16.173053 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:16.177701 11468 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:18:16.179699 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 169000us
I20260709 13:18:16.194859 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:16.195571 11543 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
W20260709 13:18:16.269877 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 107359us
W20260709 13:18:16.260560 18235 log.cc:927] Time spent T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:16.318594 11544 raft_consensus.cc:1275] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:16.332932 18240 consensus_queue.cc:1048] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:16.426385 11286 catalog_manager.cc:5028] TS 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): tablet 1c4eb2d16aa247998d0927eb34dedfa8 (table test-workload [id=feeec31c3c00468abdb13fd79d4e6c47]) successfully deleted
I20260709 13:18:16.446024 11468 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:16.451025 18246 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:16.456738 11543 raft_consensus.cc:1275] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:16.463093 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:16.468535 18246 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:16.468978 18246 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:16.469344 18246 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.476653 18246 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.478729 18246 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:16.479523 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:16.490238 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:16.495826 11473 raft_consensus.cc:2393] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 13: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:16.497795 18121 consensus_queue.cc:1048] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:16.520773 11547 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:18:16.527979 12085 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:16.528976 18121 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:16.529786 18121 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:16.530745 18121 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:16.569554 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 51984.4us
I20260709 13:18:16.573783 11293 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 12 to 13, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 13 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:16.626955 18239 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:16.627660 18239 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.678174 18239 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.678478 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:16.679399 11473 raft_consensus.cc:2393] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 19: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:16.686504 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:16.687254 12113 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 19.
I20260709 13:18:16.690557 18257 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.691102 18257 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.693169 18257 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.698935 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:16.699671 12113 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:16.701882 11441 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:16.702950 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:16.703359 18237 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:16.703728 11546 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:16.703846 18237 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.704315 18237 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:16.704905 11515 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:16.706840 17959 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:16.707257 17959 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:16.707594 17959 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:16.712982 18237 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.717882 17959 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.742844 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:16.743680 11473 raft_consensus.cc:2368] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 19. Current term is 20.
I20260709 13:18:16.744899 11514 leader_election.cc:400] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 19. Current term is 20.
I20260709 13:18:16.745239 11514 leader_election.cc:403] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:18:16.746152 18239 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:16.752887 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:16.753392 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:16.753557 12113 raft_consensus.cc:3060] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:16.760020 17959 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.762835 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:16.766403 18239 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161) with higher term. Message: Invalid argument: T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b for earlier term 19. Current term is 20.
I20260709 13:18:16.766952 12113 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 20.
I20260709 13:18:16.768337 11441 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:16.769166 18257 raft_consensus.cc:2804] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:16.769655 18257 raft_consensus.cc:697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:16.770620 18257 consensus_queue.cc:237] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.774173 11546 raft_consensus.cc:2468] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 20.
I20260709 13:18:16.776000 18237 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.798240 11293 catalog_manager.cc:5697] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 18 to 20, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 20 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:16.875190 17853 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.875797 17853 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.879729 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:16.880330 11546 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:16.881520 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:16.883142 18257 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:16.883489 18257 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.883636 17853 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.886514 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:16.883800 18257 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:16.908519 18257 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.911630 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:16.912284 11550 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:16.917279 11473 raft_consensus.cc:1240] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:16.918965 17979 consensus_queue.cc:1059] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:16.919867 18245 raft_consensus.cc:3055] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:16.920248 18245 raft_consensus.cc:740] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:16.921067 18245 consensus_queue.cc:260] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.921559 11550 raft_consensus.cc:2468] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:16.922235 18245 raft_consensus.cc:3060] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:16.923964 11442 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:16.924801 17853 raft_consensus.cc:2804] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:16.925315 17853 raft_consensus.cc:697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:16.926180 17853 consensus_queue.cc:237] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:16.930670 18257 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:16.931922 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:16.932780 12119 raft_consensus.cc:2410] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:16.941489 11293 catalog_manager.cc:5697] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 14, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.155779 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:17.211019 11550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:18:17.211629 11550 raft_consensus.cc:606] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:17.215124 11550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:18:17.215713 11550 raft_consensus.cc:606] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:17.218451 18154 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:17.218358 11548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3"
dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
mode: GRACEFUL
new_leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:51250
I20260709 13:18:17.219218 11548 raft_consensus.cc:606] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Received request to transfer leadership to TS b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:17.219148 18154 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.220727 10252 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:17.222568 10252 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:17.222967 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:17.223625 11468 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 22.
I20260709 13:18:17.223896 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:17.224637 12119 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 22: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:17.224814 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:17.225668 18258 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:18:17.225901 18154 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:17.226080 18258 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:17.226542 18258 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:18:17.230419 12119 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:18:17.232538 18257 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:18:17.238157 18258 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.241360 18258 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 22 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:17.241556 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:17.241606 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:17.242703 12113 raft_consensus.cc:2393] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 in current term 22: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:17.251040 11468 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 22.
I20260709 13:18:17.252334 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:17.253837 18154 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:18:17.260382 18246 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.260985 18246 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.268980 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:17.269626 11473 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 22.
I20260709 13:18:17.275842 18246 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:17.278672 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:17.283361 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892" is_pre_election: true
I20260709 13:18:17.289391 17979 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:17.290011 17979 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.290458 17979 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:17.316120 17979 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:17.322970 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 69062.6us
I20260709 13:18:17.323848 17979 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:17.325016 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:17.325042 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:17.327795 11473 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:17.326411 18154 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:17.332952 18154 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.346452 11548 raft_consensus.cc:1275] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:18:17.349133 18257 consensus_queue.cc:1048] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:18:17.361146 11473 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 23.
I20260709 13:18:17.364734 12083 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:17.365545 18246 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:18:17.366024 18246 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:17.366815 18246 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.373468 11293 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 reported cstate change: term changed from 21 to 22, leader changed from 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 22 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.445865 11296 catalog_manager.cc:5697] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 22 to 23, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 23 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.478495 18196 raft_consensus.cc:993] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.479101 18196 raft_consensus.cc:1081] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.480602 12119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55076
I20260709 13:18:17.481042 12119 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.481321 12119 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:17.486613 12119 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.489215 11399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:17.489809 11399 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:17.489876 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:17.490468 11548 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:17.490764 11548 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:17.491531 11548 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.492951 11548 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:17.495004 12113 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:18:17.495406 12113 pending_rounds.cc:85] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 11
I20260709 13:18:17.495635 12113 pending_rounds.cc:107] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:17.495848 12113 raft_consensus.cc:2889] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:17.497972 17853 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:17.506711 11399 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:17.507737 12085 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:17.508333 12119 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:17.508580 17979 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:17.509172 17979 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:17.510051 17979 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.511082 11548 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:17.514816 11547 raft_consensus.cc:1275] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:17.520921 18237 consensus_queue.cc:1048] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:17.548341 11293 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 15 to 16, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 16 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.621713 18196 raft_consensus.cc:993] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.622143 18196 raft_consensus.cc:1081] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.623661 12119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55076
I20260709 13:18:17.624218 12119 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.624591 12119 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:17.629721 12119 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.631472 12119 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:17.632390 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:17.632938 11548 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:17.633204 11548 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:17.632253 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:17.633945 11468 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:17.634035 11548 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.635435 11548 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:17.641499 11548 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:18:17.641563 11468 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 15.
I20260709 13:18:17.642748 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:17.643627 18246 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:17.644137 18246 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:17.644901 18246 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.654651 11293 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 14 to 15, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 15 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.666949 18283 raft_consensus.cc:993] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: : Instructing follower b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.667547 18239 raft_consensus.cc:1081] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Signalling peer b3abf4b5ee724203ad1226f5c46bfe46 to start an election
I20260709 13:18:17.669078 12119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17"
dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
 from {username='slave'} at 127.0.0.1:55076
I20260709 13:18:17.669651 12119 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.670014 12119 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.677923 12119 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.680037 12119 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:17.681165 11396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "147201d22a694ca59f6e014f083a0892"
I20260709 13:18:17.681807 11396 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.683310 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:17.683853 11547 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:17.684130 11547 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:17.684799 11547 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.685806 11547 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.690135 11396 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:18:17.691300 12085 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:17.692188 11547 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:18:17.692167 18246 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:17.693638 18246 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:17.694567 18246 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.720796 11293 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 10, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 10 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.741688 12113 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 LEADER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 22. Current term is 23. Ops: []
I20260709 13:18:17.743237 18258 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:18:17.743997 18258 consensus_queue.cc:1243] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer b3abf4b5ee724203ad1226f5c46bfe46 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:18:17.744462 18154 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:18:17.744810 18154 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:17.745611 18154 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:17.746798 18154 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:18.858728 10252 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:19.185274 11440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.002s	sys 0.001s
W20260709 13:18:19.185328 11443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.003s	sys 0.007s
W20260709 13:18:19.185998 11440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.002s	sys 0.001s
W20260709 13:18:19.186154 11443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.003s	sys 0.008s
I20260709 13:18:19.295567 18288 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:18:19.299296 11517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.000s	sys 0.013s
W20260709 13:18:19.303216 11517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.000s	sys 0.014s
I20260709 13:18:19.306234 18291 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.307876 18291 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.297163 18288 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.318089 18288 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.323118 18291 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.323202 18289 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.324959 18289 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.331794 18287 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.333861 18287 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.332734 18297 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.330727 18288 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.356535 18291 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.357338 18288 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.357676 18291 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.363478 18287 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.364460 18297 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.366308 18298 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.367426 18298 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.373502 18300 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.374593 18300 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.378086 18300 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.371762 18289 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.392287 18294 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.393189 18288 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.393561 18298 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.393220 18294 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.394913 18297 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.396286 18294 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.398327 18288 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.399261 18303 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.399176 18288 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.400207 18303 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.401407 18301 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.402269 18301 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.408905 18293 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.409885 18301 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.416553 18297 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.417492 18297 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.419180 18296 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.419934 18296 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.422327 18296 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.409940 18293 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.424559 18288 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.425098 18293 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.437822 18287 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.438834 18287 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.442090 18287 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.443477 18296 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.444295 18296 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.444752 18298 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.445680 18298 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.446353 18294 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.447293 18294 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.448282 18296 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.448870 18296 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.449431 18296 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.450253 18294 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.402945 18303 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.468945 18293 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.467500 18287 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.483661 18305 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.466499 18294 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.471637 18307 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.497125 18304 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.498445 18304 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.499145 18287 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.502029 18304 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.504467 18308 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.505501 18308 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.509712 18309 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.510531 18309 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.513276 18309 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.520185 18306 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.520625 18290 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.521170 18306 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.521346 18290 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.523537 18288 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.524103 18307 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.528782 18288 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.529214 18296 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.529847 18290 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.529964 18297 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.525148 18293 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.531610 18299 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.525071 18305 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.548358 18305 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.524935 18294 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.524711 18298 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.530661 18307 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.551262 18312 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.523711 18306 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.552237 18312 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.532388 18299 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.555083 18288 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.574371 18297 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.576320 18289 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.580461 18299 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.584347 18309 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.585093 18309 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.587489 18309 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.588392 18309 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.589067 18309 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.591075 18309 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.591658 18309 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.592458 18319 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.592406 18309 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.593417 18319 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.594756 18309 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.595415 18309 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.596244 18319 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.596091 18309 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.580652 18289 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.525308 18300 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.578856 18297 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.525352 18303 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.599195 18303 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.601444 18303 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.577430 18293 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.605842 18300 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.608341 18300 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.610231 18308 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.612679 18297 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.613579 18294 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.580649 18306 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.613857 18291 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.577729 18313 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.614619 18306 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.615612 18313 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.617020 18306 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.618036 18313 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.598076 18298 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.619515 18298 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.619921 18312 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.602839 18299 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.624679 18299 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.626106 18317 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.627178 18317 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.600502 18309 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.621922 18298 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.632474 18299 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.641733 18304 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.642606 18304 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.644934 18304 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.650975 18289 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.652770 18306 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.653518 18306 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.653995 18317 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.663445 18299 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.663918 18304 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.664378 18299 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.664641 18304 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.667467 18310 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.668187 18310 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.669629 18311 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.670318 18311 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.669116 18319 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.671422 18319 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.673645 18319 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.795123 18324 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 1
I20260709 13:18:19.828174 18306 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:19.819748 18318 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.827013 18315 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.828298 18299 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.781208 18302 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.747151 18288 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.838195 18312 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.845199 18316 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.746454 18314 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.950489 18317 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.674633 18310 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.675169 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:19.966136 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:19.969533 18322 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.845271 18294 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.714591 18300 raft_consensus.cc:493] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.675215 18292 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:18:19.997448 18327 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.997932 18292 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.998176 18327 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.998623 18330 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:19.985129 11470 raft_consensus.cc:2468] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:19.999323 18330 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.743676 18303 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.826527 18289 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:19.674167 18319 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:19.738174 18291 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.847364 18298 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.833254 18329 raft_consensus.cc:493] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.795380 18323 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.012184 11365 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:20.011965 18323 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.014302 18325 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.827750 18313 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.015950 18332 raft_consensus.cc:2804] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:19.745121 18295 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:19.674165 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.019762 18295 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.020920 18331 raft_consensus.cc:493] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.021651 18331 raft_consensus.cc:515] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.022126 18295 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.019791 18291 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.023079 18324 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.024010 18331 leader_election.cc:290] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.024466 18302 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.958425 18311 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.014685 18323 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:19.995791 18294 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.996835 18300 raft_consensus.cc:515] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.985074 11473 raft_consensus.cc:2468] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 9.
I20260709 13:18:20.010605 18327 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.010368 18318 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.011188 18310 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.010927 18303 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.015431 18313 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.674903 18320 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.015345 18288 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.016728 18304 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.016434 18314 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.017277 18317 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.017982 18289 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.018913 18326 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.015281 10252 tablet_server.cc:179] TabletServer@127.10.3.1:0 shutting down...
I20260709 13:18:20.023754 18298 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.024924 18291 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:19.994732 18322 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:19.995060 18315 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:19.989709 11396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41730 (request call id 3478) took 2170 ms. Trace:
I20260709 13:18:19.827976 18293 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.019429 18306 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.019291 18330 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.063189 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.066169 18287 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.037461 18332 raft_consensus.cc:493] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.066243 18310 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.069473 18313 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.069906 18329 raft_consensus.cc:515] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.070554 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.070688 18328 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.071305 18336 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.072038 18294 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.072863 18318 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.070712 18319 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.083678 11365 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:20.084499 18323 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.026006 18324 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.073355 18327 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.098269 18333 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.100063 18301 raft_consensus.cc:493] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.105886 18337 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.107069 18322 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.064247 18312 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.108563 18315 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.091084 11396 rpcz_store.cc:276] 0709 13:18:17.815819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.816362 (+   543us) service_pool.cc:224] Handling call
0709 13:18:19.968558 (+2152196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.968949 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.968992 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.972436 (+  3444us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:18:19.972469 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.972510 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.986486 (+ 13976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:20.123483 18288 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.089006 18303 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.080838 18320 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.155542 18314 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.088693 18309 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.156371 18309 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.156528 18323 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.157313 18320 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.015276 18325 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.158031 18289 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.158528 18309 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.104701 18332 raft_consensus.cc:3060] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.159391 18325 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.159101 18327 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.065122 18316 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.161232 18327 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.162133 18316 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.161857 18333 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.163657 18302 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.165328 18317 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.165629 18301 raft_consensus.cc:515] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.166970 18298 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.167528 18298 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.167657 18319 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.149457 18293 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.168246 18298 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.168972 11469 raft_consensus.cc:2468] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:20.155197 18337 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.152429 18336 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.158793 18323 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.138960 18330 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.155247 18312 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.165361 18329 leader_election.cc:290] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.141139 18341 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.144943 18306 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.167778 18301 leader_election.cc:290] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.147353 18326 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:20.149461 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 65858us
I20260709 13:18:20.150373 18328 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.185132 18340 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.186471 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.187383 11550 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:20.178704 18310 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.180936 18317 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.193490 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.185972 18331 raft_consensus.cc:2804] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.187314 18340 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.196013 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.197333 18342 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.197422 18323 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.199157 18326 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.199242 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.199909 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.200008 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.201666 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.201817 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.203250 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.204035 18337 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.204823 18300 leader_election.cc:290] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.205526 18293 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.231683 18298 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.231969 18328 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.194294 18330 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.232201 18317 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.232810 18332 raft_consensus.cc:515] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.234117 18341 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.237010 11544 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:18:20.240191 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.244237 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.195839 18331 raft_consensus.cc:493] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.194051 18344 raft_consensus.cc:493] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.244856 11544 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:20.246133 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.241919 11470 raft_consensus.cc:1275] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:18:20.246802 18298 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.247715 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.248171 18330 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.248627 18300 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.249075 18320 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.250303 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.251484 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.253003 18342 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.254290 18306 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.255069 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.255210 18336 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.241176 18343 raft_consensus.cc:493] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.255959 18333 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.255757 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.240051 18323 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.257176 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.297236 11549 raft_consensus.cc:2468] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:20.298280 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.298583 18310 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.299049 11475 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 9.
I20260709 13:18:20.299401 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.299259 18310 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.300022 11542 raft_consensus.cc:2468] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:20.300216 11477 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 19.
I20260709 13:18:20.300722 11474 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:20.301175 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.301321 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.301399 18310 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.301744 11542 raft_consensus.cc:2468] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:18:20.298280 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.302492 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.289980 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.303563 18337 raft_consensus.cc:493] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.303369 18343 raft_consensus.cc:515] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.304209 18310 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.304178 18337 raft_consensus.cc:515] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.304618 18345 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.304998 18310 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.305338 18345 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.306442 18300 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.307497 18345 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.308033 18345 raft_consensus.cc:493] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.308790 18300 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.289201 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.291615 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.309747 11548 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:18:20.310205 11546 raft_consensus.cc:2468] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
I20260709 13:18:20.310634 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.311327 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.311975 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.312747 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.313117 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.243920 11545 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:20.301545 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.308516 18345 raft_consensus.cc:515] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.314904 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.314754 18320 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.315593 11545 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 11.
I20260709 13:18:20.316561 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.316649 18345 leader_election.cc:290] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.317003 18320 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.317281 18339 raft_consensus.cc:493] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.315187 18299 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.305732 18343 leader_election.cc:290] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.287791 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.255777 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.302645 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.314079 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.304337 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.319788 11550 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 18.
I20260709 13:18:20.317900 18339 raft_consensus.cc:515] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.318076 18299 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.321834 18298 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.195545 18327 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:20.322872 18327 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.314787 11474 raft_consensus.cc:2468] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 16.
I20260709 13:18:20.323489 18339 leader_election.cc:290] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.323729 18299 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.324206 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.324602 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.325042 18327 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.325414 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.299957 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.314160 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.325657 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.318791 11549 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:18:20.460327 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.319774 11542 raft_consensus.cc:2468] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:20.458508 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.319885 11547 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 15.
I20260709 13:18:20.472398 11543 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:18:20.476312 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.474874 11473 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:18:20.476930 11474 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
W20260709 13:18:20.471839 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 157870us
I20260709 13:18:20.480175 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.484143 18325 raft_consensus.cc:493] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:20.484838 18325 raft_consensus.cc:515] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.472982 18348 raft_consensus.cc:493] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.487658 18348 raft_consensus.cc:515] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.489852 11550 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:20.490499 11470 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:20.459336 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.491508 11472 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:18:20.331511 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.481647 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.326020 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:20.439582 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080035e160 after lost signal to thread 10255
I20260709 13:18:20.507114 11365 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:20.320369 18342 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.508126 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.481616 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.473078 18298 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.250319 18332 leader_election.cc:290] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.298353 18323 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.477614 18331 raft_consensus.cc:3060] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.513382 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.327579 18337 leader_election.cc:290] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.507083 11551 raft_consensus.cc:2468] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:20.337342 18341 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:20.514815 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08000d18e0 after lost signal to thread 11334
I20260709 13:18:20.339177 18317 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.457695 18340 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.274461 18292 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.325933 18310 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.515949 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.516664 18349 raft_consensus.cc:2804] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:20.522519 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.523232 18349 raft_consensus.cc:493] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.527419 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.530345 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.532684 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.530447 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.533002 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.533196 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.533558 11477 raft_consensus.cc:2468] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 10.
I20260709 13:18:20.533648 12115 raft_consensus.cc:2468] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:18:20.533908 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.531558 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.534503 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.534924 12116 raft_consensus.cc:2468] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 18.
I20260709 13:18:20.535559 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.531515 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.531840 11514 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.531922 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.536441 12119 raft_consensus.cc:2468] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 22.
I20260709 13:18:20.532343 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.536401 11469 raft_consensus.cc:2468] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 11.
I20260709 13:18:20.535811 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:20.530822 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080037ec80 after lost signal to thread 11426
I20260709 13:18:20.538858 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.526157 18349 raft_consensus.cc:3060] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:20.534591 11475 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 15.
I20260709 13:18:20.538774 11477 raft_consensus.cc:2468] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:18:20.541759 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.543286 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.545171 11546 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 19.
W20260709 13:18:20.547894 12144 debug-util.cc:398] Leaking SignalData structure 0x7b080032fea0 after lost signal to thread 11501
I20260709 13:18:20.548023 18320 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:20.548556 18320 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.547943 18348 leader_election.cc:290] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.549065 18320 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.544102 18292 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:20.549587 12144 debug-util.cc:398] Leaking SignalData structure 0x7b08003bdb40 after lost signal to thread 11575
I20260709 13:18:20.549641 18292 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.550294 18292 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:20.542855 18343 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:20.551110 12144 debug-util.cc:398] Leaking SignalData structure 0x7b0800386c40 after lost signal to thread 12147
I20260709 13:18:20.551370 18343 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.551769 18343 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.554694 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ff2b5fd08d4882a5355cf7758991cf" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.555958 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.556954 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.557768 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.558403 12119 raft_consensus.cc:2468] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 15.
I20260709 13:18:20.558382 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.559124 12116 raft_consensus.cc:2468] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 19.
I20260709 13:18:20.559214 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.559508 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.559985 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.560964 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.557804 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.562085 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.561144 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.564329 11548 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:20.566673 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.567378 12119 raft_consensus.cc:2468] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:20.560004 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.569046 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.569607 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.570211 12116 raft_consensus.cc:2468] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 20.
I20260709 13:18:20.571026 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.571312 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:20.568933 12118 raft_consensus.cc:2468] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 14.
I20260709 13:18:20.558382 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.572387 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.574182 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.574796 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.574870 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.575321 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.569142 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.576153 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.576825 18293 raft_consensus.cc:2804] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:20.577215 18293 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.577461 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.577736 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.577214 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.578431 12115 raft_consensus.cc:2468] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:18:20.578522 11473 raft_consensus.cc:2468] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:18:20.577605 18293 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.583846 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.586794 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.587050 18325 leader_election.cc:290] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.588167 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.588778 12117 raft_consensus.cc:2468] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:20.589895 11475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:20.590587 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3fd80658464474dbe463b1943becc75" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.591853 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52a477150c345c8ba80211d1a49b41c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.593369 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.594498 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.325661 18344 raft_consensus.cc:515] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.597123 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.598215 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.598714 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.597681 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.599808 11542 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:18:20.599565 18316 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:20.604043 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.604355 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.605302 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.605772 11550 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 10.
I20260709 13:18:20.607149 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.608867 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66cb4598d33043fc8eb235647a242814" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.614470 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.610706 18322 raft_consensus.cc:2804] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:20.605069 18316 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.616027 11514 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.603706 18292 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.616604 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.618937 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.619761 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.619482 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe5740695024e83afc5069cfb41118c" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.620808 11514 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.621742 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.608439 18343 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.624286 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.617295 18314 raft_consensus.cc:2804] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.616600 18322 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.617554 18316 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.623471 11545 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:20.626214 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.619428 18333 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.625669 18314 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.629506 18314 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:20.627039 18312 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:20.630102 18312 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.630517 18312 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.621726 18289 raft_consensus.cc:2804] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:20.628755 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.628640 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.631764 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.628126 11514 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.636407 11548 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:20.634955 12123 raft_consensus.cc:2468] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 16.
I20260709 13:18:20.637773 11442 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.627053 18300 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:20.627096 18322 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:20.639312 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:20.638798 18315 raft_consensus.cc:2804] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:20.628733 18333 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.643306 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.639354 11442 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.635037 18289 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.640622 18294 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.641876 11367 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.647408 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a7fd6f89ac4da3ab7769d1379f5bb8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.647840 18292 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.639019 18334 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.640650 18300 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.641431 18315 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:20.645110 18333 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:20.654844 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:20.655771 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 54959.8us
I20260709 13:18:20.648814 18289 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.648808 18294 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.661880 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.666028 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.666038 18349 raft_consensus.cc:515] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.666914 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.668226 12123 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:20.669582 18344 leader_election.cc:290] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.669675 18349 leader_election.cc:290] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.653429 18300 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.663898 18294 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:20.654387 18315 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:20.652240 18334 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.659837 18325 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:20.678081 11442 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.674394 18334 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:20.678795 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.679925 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.692806 11471 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 23.
I20260709 13:18:20.688476 18316 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.663787 18293 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.695547 11367 leader_election.cc:304] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.705986 18322 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.707103 18325 raft_consensus.cc:2749] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:20.734159 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.711935 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48129c3807a242fca6439f00aefd9b22" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.735219 18317 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:20.714321 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:20.736572 18293 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.736996 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfd1f78aa1d843c28021b808cf5fd960" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.737207 18293 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:20.737658 18293 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.738018 18293 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.735234 11470 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:20.738302 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.713195 18320 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.739439 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.738214 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c4eb2d16aa247998d0927eb34dedfa8" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.740621 11547 raft_consensus.cc:1240] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:20.741036 18320 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.716545 18300 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.717563 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b39fd56a8d45f7b56a9ac114b66d2b" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:20.719869 18312 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.742066 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.742802 12123 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:20.743283 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.745537 18312 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.754180 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.754679 18300 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.722693 18343 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:20.725473 18315 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.727960 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.756439 12117 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 16.
I20260709 13:18:20.733075 18333 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.734828 12112 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 17.
I20260709 13:18:20.740783 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.758057 12115 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 9.
I20260709 13:18:20.759868 18333 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.760878 11472 raft_consensus.cc:1240] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 19. Current term is 20. Ops: []
I20260709 13:18:20.761786 18315 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.762796 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.779517 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.781291 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.782796 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
W20260709 13:18:20.838879 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 160852us
I20260709 13:18:20.843779 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.740986 18294 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.720422 18331 raft_consensus.cc:515] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:20.747275 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 52583.8us
I20260709 13:18:20.748131 18316 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.749895 18322 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.751030 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:20.716607 18314 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.751396 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.895586 12119 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 13.
I20260709 13:18:20.896763 18314 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.896996 18331 leader_election.cc:290] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.897765 11367 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.751552 18289 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:20.752621 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.753533 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:20.755089 18312 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:20.721849 18334 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.946655 11442 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.946776 18334 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.948523 18312 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.950111 18312 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:20.949635 11442 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.953380 11442 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.953248 18293 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.954818 18315 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.955269 18315 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.955665 18315 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.956637 18289 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:20.958093 18294 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.948709 18316 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:20.954309 18351 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:20.959288 18312 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:20.954264 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
W20260709 13:18:20.964010 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 78073.8us
W20260709 13:18:20.960417 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 209524us
I20260709 13:18:20.967592 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.968276 18298 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:20.968353 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:20.969954 18298 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:20.970113 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:20.970412 18298 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.971141 18340 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:20.971043 18317 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:20.972798 11367 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.975980 18293 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.965425 18351 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.994930 18351 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:20.995985 11367 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.997025 18310 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:20.998260 18348 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:20.998976 18348 consensus_queue.cc:1059] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:20.999982 18348 raft_consensus.cc:3055] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:18:20.964349 18325 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:18:21.000800 18325 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.001288 18325 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:21.002099 11470 raft_consensus.cc:1240] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:20.978438 18312 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:20.978681 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:20.965076 18316 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b3abf4b5ee724203ad1226f5c46bfe46)
I20260709 13:18:20.987767 11471 raft_consensus.cc:1240] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:20.988346 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.000298 18348 raft_consensus.cc:740] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:20.989663 18298 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:20.998433 11367 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:20.988957 18340 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.991453 18294 raft_consensus.cc:2804] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:21.013554 11514 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.009298 18316 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:21.025975 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.024288 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.028111 11514 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.030401 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.016168 18293 raft_consensus.cc:2804] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:21.017390 18340 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.038308 18293 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.047680 18293 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.035595 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3205d9e6dce4aad891fa90943f21879" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.019732 18331 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.051532 18317 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:21.052062 18317 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.052511 18317 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:21.019256 18344 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:18:21.056021 18344 raft_consensus.cc:3055] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:21.056418 18344 raft_consensus.cc:740] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.056473 11476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.057307 18344 consensus_queue.cc:260] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.055980 18315 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.041391 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.060099 11542 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 17: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
W20260709 13:18:21.059979 11516 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.3.1:37459 timed out after 1.527s (SENT) [suppressed 1 similar messages]
I20260709 13:18:21.041468 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.041620 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95e3d59427ee46f989532eb3cdc311cc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.041391 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.042197 18349 consensus_queue.cc:1059] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:21.061064 18315 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.061152 18331 consensus_queue.cc:1059] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:21.036610 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bea8de48c2942d99a1477b5b427be03" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.062119 18345 raft_consensus.cc:3055] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:21.062530 18345 raft_consensus.cc:740] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.018621 18294 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.063548 18294 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:21.063342 18345 consensus_queue.cc:260] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.042065 18310 consensus_queue.cc:1059] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:18:21.064976 18310 raft_consensus.cc:3055] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:21.065410 18310 raft_consensus.cc:740] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.034708 18300 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:21.066735 18300 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.067236 18300 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.069471 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.070655 11470 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 20: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:21.044126 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.058061 18348 consensus_queue.cc:260] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.042924 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f946b7973fe482681135635493ebd48" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.071990 18348 raft_consensus.cc:3060] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:21.072471 11550 raft_consensus.cc:2393] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 20: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:21.073383 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.074474 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.034561 18343 raft_consensus.cc:2804] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:21.075186 18343 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.075877 18343 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:21.075071 18346 consensus_queue.cc:1175] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34fe1737ebb840fb9746980f649cc0a6" responder_term: 20 status { last_received { term: 19 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:21.023778 18334 raft_consensus.cc:2804] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:21.079411 18334 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.079545 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.079864 18334 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.063797 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5de422ef4884189bea49cec7b7c2ec1" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.066330 18310 consensus_queue.cc:260] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.092418 18310 raft_consensus.cc:3060] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:21.098075 11477 raft_consensus.cc:2393] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 17: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
W20260709 13:18:21.141877 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 236245us
I20260709 13:18:21.143002 12083 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.144188 12083 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.147522 18323 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:21.148330 18323 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.148758 18323 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.149186 18323 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.148977 18316 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.150512 18351 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.151554 18334 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.160280 18323 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.163283 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.164790 18317 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.176820 18325 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.180508 18340 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.186374 18294 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.197161 18343 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.191844 18300 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.231916 11474 raft_consensus.cc:1240] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:21.234277 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 267916us
W20260709 13:18:21.239140 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 204791us
I20260709 13:18:21.139945 18293 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:21.240033 11367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 205575us
W20260709 13:18:21.242653 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 182809us
I20260709 13:18:21.246001 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:21.250180 11365 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.248857 18310 consensus_queue.cc:1059] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:18:21.268488 18352 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:21.276087 18300 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:21.286823 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 87907.5us
I20260709 13:18:21.241110 18298 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:21.313916 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.3:46781, user_credentials={real_user=slave}} blocked reactor thread for 46719.4us
W20260709 13:18:21.316325 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 170648us
W20260709 13:18:21.315642 11365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 61492.3us
I20260709 13:18:21.265324 18353 consensus_queue.cc:1048] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 13:18:21.317242 12083 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.323251 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.324031 11472 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:18:21.328698 11477 raft_consensus.cc:1240] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.312793 18325 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 24 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.330818 18340 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.331154 18323 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:21.329344 11468 raft_consensus.cc:1240] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:21.315299 18316 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.332118 18298 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.328516 18351 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.335122 18354 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.335399 12083 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.336105 18293 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:21.336179 18323 raft_consensus.cc:2749] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.337260 18317 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:21.337749 12083 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:21.338608 18353 raft_consensus.cc:2804] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:21.338697 12083 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.338865 18294 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:21.339069 18353 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:21.339521 18353 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.340516 18361 consensus_queue.cc:1059] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.341647 12083 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.342670 18352 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.352337 18331 raft_consensus.cc:3055] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:21.352800 18331 raft_consensus.cc:740] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.353631 18331 consensus_queue.cc:260] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.343294 12083 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:21.344014 18344 raft_consensus.cc:3060] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.354822 18331 raft_consensus.cc:3060] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.344573 18325 raft_consensus.cc:3055] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:21.333081 18358 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.345146 18323 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.347456 18343 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:21.348471 18334 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:21.340845 18345 raft_consensus.cc:3060] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.358428 18325 raft_consensus.cc:740] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.363894 18325 consensus_queue.cc:260] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.364547 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.365190 11550 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.341586 18361 consensus_queue.cc:1059] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:21.361567 18363 raft_consensus.cc:2804] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:21.359043 12083 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.369779 12083 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.371891 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.372571 11551 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.367920 12085 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.377844 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.378449 11550 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 11.
I20260709 13:18:21.375288 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.369092 18363 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:18:21.375240 11516 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.379374 11474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b22c9e294a846159d4831b92f692943" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.365358 18325 raft_consensus.cc:3060] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:21.381297 11477 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 16: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:21.382036 11473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa999fcc28454bc88f93770d07ff72e2" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.383517 11546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e0fc54e8b645deb82c6013c5d5c7ea" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.385594 11546 raft_consensus.cc:2393] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 18: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:21.384756 11548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.379623 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.387523 11548 raft_consensus.cc:2393] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 16: Already voted for candidate 9e9c2603c0bd4d25b713649b3890c46b in this term.
I20260709 13:18:21.387588 11545 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.387744 11470 raft_consensus.cc:1240] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:21.385667 11367 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.375103 18354 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.377794 18358 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.379297 18352 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:21.390038 11550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255638a15529487e8ade55aaaa8455d5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.391713 11472 raft_consensus.cc:2468] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 24.
I20260709 13:18:21.393059 11549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.394255 11547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.394485 11544 raft_consensus.cc:1240] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:18:21.380077 18363 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:21.396833 11516 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.394048 11366 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.398341 11516 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.399068 11542 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:21.400674 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.410363 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930d965be7eb49448fa23456beb9925a" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.411391 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.412297 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.401333 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.401926 12117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.402599 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b37b6683af04823b0814ba56c4ea803" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.403385 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.404767 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e9e663b46c5417b9a0e42fdc7b70969" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.406414 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058fa5ec3edb42c796103c7361c1a8e1" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.406684 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d02abb4f52449679a2415346f354340" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.407258 18343 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.419431 18349 consensus_queue.cc:1059] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:21.419595 12112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.420449 18349 raft_consensus.cc:3055] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:21.420805 18349 raft_consensus.cc:740] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.421547 18349 consensus_queue.cc:260] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.408380 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4de37bdc334587a90c03b40bfd6c5f" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.393093 18298 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:21.423336 18298 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.424268 18298 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.408689 12114 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.415532 11551 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147201d22a694ca59f6e014f083a0892 in term 17.
I20260709 13:18:21.426079 18360 raft_consensus.cc:3055] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:21.426498 18360 raft_consensus.cc:740] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.426872 11543 raft_consensus.cc:1240] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:18:21.426846 11367 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b; no voters: 
I20260709 13:18:21.427318 18360 consensus_queue.cc:260] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.428148 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.428721 18360 raft_consensus.cc:3060] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.428869 18310 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.429256 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.417289 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:21.429607 18310 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:21.429803 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.430008 18361 raft_consensus.cc:2804] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:21.431617 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.417598 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.416070 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.422835 18349 raft_consensus.cc:3060] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:21.435365 11516 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.435840 11366 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.439142 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.439510 18355 raft_consensus.cc:2804] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:18:21.435348 11365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6; no voters: 
I20260709 13:18:21.430508 18340 raft_consensus.cc:2804] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:21.439431 10252 tablet_replica.cc:333] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.442108 18340 raft_consensus.cc:493] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.442698 18340 raft_consensus.cc:3060] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:21.443193 10252 raft_consensus.cc:2243] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.441964 18348 raft_consensus.cc:2804] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:18:21.443897 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:21.444886 11471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
W20260709 13:18:21.445053 11516 consensus_peers.cc:597] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.445667 11516 consensus_peers.cc:597] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.446210 11516 consensus_peers.cc:597] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.446750 11516 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.447045 18356 raft_consensus.cc:2804] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:21.447455 18356 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
W20260709 13:18:21.447608 11516 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.447872 18356 raft_consensus.cc:3060] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:21.448244 11516 consensus_peers.cc:597] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.448856 11516 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.449208 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.450266 18334 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.430527 18361 raft_consensus.cc:493] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:21.451658 11516 leader_election.cc:336] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.430197 18310 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.452157 11516 leader_election.cc:304] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.442175 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.451079 18361 raft_consensus.cc:3060] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.455461 18348 raft_consensus.cc:697] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 LEADER]: Becoming Leader. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.458953 11471 raft_consensus.cc:1240] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:21.459061 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.440306 18343 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:21.441613 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.460829 10252 raft_consensus.cc:2272] T 66cb4598d33043fc8eb235647a242814 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.460964 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.461656 11470 raft_consensus.cc:1240] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 24 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:18:21.477516 11441 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.439950 18355 raft_consensus.cc:493] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.479209 18355 raft_consensus.cc:3060] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:18:21.478974 18310 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.479157 18294 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.480206 18294 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.480556 18294 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.484251 18340 raft_consensus.cc:515] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.462538 18334 raft_consensus.cc:2749] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.464268 18353 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.465621 18348 consensus_queue.cc:237] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.460448 12114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.466969 11468 raft_consensus.cc:1240] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:18:21.485507 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 43974.5us
I20260709 13:18:21.522949 12122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.486027 18346 raft_consensus.cc:2804] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:18:21.489385 10252 tablet_replica.cc:333] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.470894 18363 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.512503 11441 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.527549 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.525943 11366 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.521981 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.461766 11516 consensus_peers.cc:597] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.524938 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.525158 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.528998 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 67285.8us
I20260709 13:18:21.529587 12116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.528013 11441 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.525290 10252 raft_consensus.cc:2243] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.524786 18346 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:21.524928 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.531411 12085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 140266us
I20260709 13:18:21.531795 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.533847 11470 raft_consensus.cc:1240] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:21.529676 11516 consensus_peers.cc:597] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.536022 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:21.537133 11516 consensus_peers.cc:597] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.519294 18356 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.537616 11366 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.538203 11441 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:21.578641 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 43830us
W20260709 13:18:21.580344 11516 leader_election.cc:336] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.533459 18289 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.534606 18346 raft_consensus.cc:3060] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:21.585383 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 47850.2us
I20260709 13:18:21.586010 11366 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:21.586644 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 48491.8us
W20260709 13:18:21.581871 11516 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.587502 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.587553 11441 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.590708 11441 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.591550 10252 raft_consensus.cc:2272] T 8e90063696c0437bb8ad2d545f45a9ce P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.540515 18344 raft_consensus.cc:2804] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:21.538908 18298 raft_consensus.cc:2804] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:21.537205 18317 consensus_queue.cc:1059] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:18:21.596479 18298 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:21.536988 18345 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.597146 18317 consensus_queue.cc:1059] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.541070 18312 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:21.586794 18289 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.594301 18354 consensus_queue.cc:1059] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 13:18:21.610472 18315 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:21.593896 18362 raft_consensus.cc:2804] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:21.646222 18315 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:21.651037 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 53937.5us
I20260709 13:18:21.587280 18331 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.591606 18310 raft_consensus.cc:2804] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:21.607249 10252 tablet_replica.cc:333] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.591630 18293 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:21.613667 18316 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.655723 18293 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.655843 18316 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.656185 18293 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:21.656265 18316 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.659291 18354 raft_consensus.cc:3055] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:18:21.656286 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d2059876e34ff89bf967e96ab7bf09" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.596954 18298 raft_consensus.cc:3060] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:21.632682 18356 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.634048 18340 leader_election.cc:290] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.635586 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.636430 18363 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:21.575670 18361 raft_consensus.cc:515] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.640343 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.630860 18323 raft_consensus.cc:3055] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.665544 18323 raft_consensus.cc:740] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.665844 18361 leader_election.cc:290] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.640550 18294 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.647272 18362 raft_consensus.cc:493] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.666399 18323 consensus_queue.cc:260] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.625613 18312 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.667461 10252 raft_consensus.cc:2243] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.667515 11472 raft_consensus.cc:1240] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:21.667721 11442 leader_election.cc:336] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.667622 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.600963 18353 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:21.595700 18344 raft_consensus.cc:493] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.672856 18344 raft_consensus.cc:3060] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:18:21.670101 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e106bf87136140b4bebad713d83d02fe" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.669343 18312 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.659699 18354 raft_consensus.cc:740] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.666891 18362 raft_consensus.cc:3060] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.671164 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b61ac66d97f84c1186c6e2b1d79477b2" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.653468 18315 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.670298 18369 raft_consensus.cc:3055] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:21.681020 18369 raft_consensus.cc:740] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.681985 18369 consensus_queue.cc:260] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.681284 18294 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.683583 18354 consensus_queue.cc:260] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.684867 18354 raft_consensus.cc:3060] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:18:21.624445 11516 leader_election.cc:336] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.688221 11516 leader_election.cc:304] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.644874 18355 raft_consensus.cc:515] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:21.668622 11442 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.671952 18323 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.694645 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.676471 18293 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.668828 18310 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:21.692286 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e8ed42aaa39460bbea1f8deeccd0adc" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.627389 18289 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.693174 11515 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.712267 18312 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.713184 11515 leader_election.cc:304] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.713892 12083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.2:36161, user_credentials={real_user=slave}} blocked reactor thread for 134318us
W20260709 13:18:21.694940 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 106232us
W20260709 13:18:21.695870 11516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 96352.1us
I20260709 13:18:21.697293 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.700366 11545 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.701216 18355 leader_election.cc:290] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 23 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.686329 18298 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.701400 18300 raft_consensus.cc:2749] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.704149 18316 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.704444 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.716559 18298 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.717207 11468 raft_consensus.cc:1240] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.717666 11470 raft_consensus.cc:1240] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.717721 18316 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.705636 18369 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:21.692893 18362 raft_consensus.cc:515] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.710426 18310 raft_consensus.cc:3060] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.713991 18334 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.718735 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.718807 11366 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.720548 11515 leader_election.cc:304] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.721552 11516 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.721882 11472 raft_consensus.cc:1240] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:21.714506 18333 raft_consensus.cc:2749] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.722883 18298 raft_consensus.cc:2804] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:21.722121 11516 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.723222 11470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ebf4312af74188830a07342d0dc119" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:21.723845 11516 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.724247 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.725275 11366 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.725466 18333 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.725713 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44539667cbc04657b801ff659dee5bde" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:21.725250 11516 leader_election.cc:336] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.726696 11516 leader_election.cc:336] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.723394 18298 raft_consensus.cc:493] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:21.727104 11516 leader_election.cc:304] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.727288 18298 raft_consensus.cc:3060] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:21.728193 11516 leader_election.cc:336] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.728624 11516 leader_election.cc:304] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.729365 18334 raft_consensus.cc:2749] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.730013 18334 raft_consensus.cc:2749] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:21.731069 11516 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.731509 18355 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.712116 18293 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.734159 11515 leader_election.cc:304] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.734896 18334 raft_consensus.cc:2749] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:21.735101 18312 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.735653 18334 raft_consensus.cc:2749] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.736377 11515 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.737134 18334 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.737459 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.738699 11472 raft_consensus.cc:1240] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Rejecting Update request from peer b3abf4b5ee724203ad1226f5c46bfe46 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:21.739125 11515 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.739892 18300 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.740366 18315 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.744975 12113 raft_consensus.cc:2410] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 147201d22a694ca59f6e014f083a0892 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:18:21.746152 18374 consensus_queue.cc:1059] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:21.741513 18353 consensus_queue.cc:1059] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.747244 18371 raft_consensus.cc:3055] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:21.747581 11516 consensus_peers.cc:597] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.747684 18371 raft_consensus.cc:740] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.747807 18363 raft_consensus.cc:3055] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:21.748262 11516 leader_election.cc:336] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.748286 18363 raft_consensus.cc:740] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.748700 11516 leader_election.cc:304] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.748631 18371 consensus_queue.cc:260] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.750352 18371 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.750440 18363 consensus_queue.cc:260] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.751312 10252 raft_consensus.cc:2272] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.751853 18363 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.754494 10252 tablet_replica.cc:333] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.757321 18346 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.758379 18344 raft_consensus.cc:515] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.741783 18317 consensus_queue.cc:1059] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:21.760071 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.760545 18317 raft_consensus.cc:3055] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:21.760967 18317 raft_consensus.cc:740] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.761919 18317 consensus_queue.cc:260] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.763267 18317 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:21.735621 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.743516 18298 raft_consensus.cc:515] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.767766 18355 raft_consensus.cc:2804] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:21.768186 18355 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:21.768539 18355 raft_consensus.cc:3060] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.741678 18323 consensus_queue.cc:1059] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:18:21.770813 18323 raft_consensus.cc:3055] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:18:21.771231 18323 raft_consensus.cc:740] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.772231 18298 leader_election.cc:290] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:21.772804 11441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 182158us
I20260709 13:18:21.774155 11441 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.775350 11441 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.776093 18312 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.776685 18312 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.777454 18355 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.741539 18362 leader_election.cc:290] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.741112 11515 leader_election.cc:304] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.758422 18289 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.779726 11471 raft_consensus.cc:1240] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:21.780076 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
W20260709 13:18:21.694741 11442 leader_election.cc:336] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.781723 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 87018.9us
W20260709 13:18:21.782318 11442 consensus_peers.cc:597] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.783314 11366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 47776.3us
I20260709 13:18:21.772261 18323 consensus_queue.cc:260] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.784271 18323 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:21.787451 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:21.788690 11551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:18:21.782851 11442 consensus_peers.cc:597] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.789606 11442 consensus_peers.cc:597] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.790156 11442 consensus_peers.cc:597] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.790689 11442 consensus_peers.cc:597] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.791523 11516 consensus_peers.cc:597] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.791982 18333 raft_consensus.cc:2749] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:18:21.792220 11516 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.792660 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.793419 18333 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:18:21.785112 12085 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.795197 12085 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.795994 18374 raft_consensus.cc:2749] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.797333 18374 consensus_queue.cc:1059] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.798277 18354 raft_consensus.cc:3055] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:21.798659 18354 raft_consensus.cc:740] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.799571 18354 consensus_queue.cc:260] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.801028 18355 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.784446 11366 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.802441 11366 leader_election.cc:304] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.802676 18325 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.803423 18325 raft_consensus.cc:2749] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.746398 18315 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.785597 18300 raft_consensus.cc:2749] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:21.786227 18344 leader_election.cc:290] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:21.791455 11515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.4:45993, user_credentials={real_user=slave}} blocked reactor thread for 50430.7us
I20260709 13:18:21.804380 11366 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.804903 11441 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.805723 18315 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.784897 18310 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.806149 11441 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.806298 18300 consensus_queue.cc:1059] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:21.807289 11441 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.807593 18333 raft_consensus.cc:3055] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.808035 18373 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.809178 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:21.808210 18333 raft_consensus.cc:740] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:21.809425 11441 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.810369 11442 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.810211 18333 consensus_queue.cc:260] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.807391 18315 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.811528 18333 raft_consensus.cc:3060] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.811748 18346 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:21.812796 11516 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.813673 11551 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:21.814723 18354 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.827275 11516 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.785166 18289 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Requested vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:21.828359 11516 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.828758 11516 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.829449 18300 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:21.829738 11516 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.830164 11516 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.830183 18300 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.830880 18366 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:21.831215 11514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.829049 18355 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:18:21.832123 11366 leader_election.cc:304] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.832866 18344 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.833244 11366 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.822801 12085 consensus_peers.cc:597] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.833575 18344 raft_consensus.cc:2749] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:21.834301 12085 consensus_peers.cc:597] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.834930 12085 consensus_peers.cc:597] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.835752 18298 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.814761 18310 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.809455 11515 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.839250 11515 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.816962 18374 consensus_queue.cc:1059] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:21.823382 10252 raft_consensus.cc:2243] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:21.839825 12085 consensus_peers.cc:597] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.840224 10252 raft_consensus.cc:2272] T 95e3d59427ee46f989532eb3cdc311cc P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.840443 12085 consensus_peers.cc:597] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.840758 18374 raft_consensus.cc:3055] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:18:21.840996 12085 leader_election.cc:336] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.841200 18374 raft_consensus.cc:740] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:21.841449 12085 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:21.842160 11516 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.842118 18374 consensus_queue.cc:260] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.824218 11441 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.847596 18289 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:21.847675 12085 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.848161 12085 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.823835 18373 raft_consensus.cc:2749] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.848744 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba703fc47843af924d4e45f12db16b" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.848958 18354 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.850498 18366 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:21.851555 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.852177 12115 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
W20260709 13:18:21.852795 12085 leader_election.cc:336] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.853082 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.853232 12085 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.853721 12113 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 19.
W20260709 13:18:21.854262 12085 leader_election.cc:336] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.854677 12085 leader_election.cc:304] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.854684 18323 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:18:21.855691 18323 raft_consensus.cc:2749] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.844475 10252 tablet_replica.cc:333] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
W20260709 13:18:21.855792 12085 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.857714 12085 consensus_peers.cc:597] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.846638 18374 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:21.842713 11516 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.843299 18334 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.845474 11515 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.859127 18334 raft_consensus.cc:2749] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.810804 11442 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.857842 11468 raft_consensus.cc:1240] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:21.860615 11515 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.861382 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51042.4us
I20260709 13:18:21.861641 11515 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.861989 11442 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.846549 18371 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.863198 18373 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.863523 18334 raft_consensus.cc:2749] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:21.862775 11442 consensus_peers.cc:597] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.864219 11442 consensus_peers.cc:597] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.864192 18334 raft_consensus.cc:2749] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.865023 18355 consensus_queue.cc:1059] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:21.862672 11515 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.865914 11442 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.866400 11515 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.866715 11442 leader_election.cc:336] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.867158 11442 leader_election.cc:304] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.867270 12085 leader_election.cc:336] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.867496 18334 raft_consensus.cc:2749] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.867964 12085 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.867189 18366 raft_consensus.cc:2749] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.868391 18346 raft_consensus.cc:3055] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.868788 18346 raft_consensus.cc:740] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:21.869802 18374 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.869624 18346 consensus_queue.cc:260] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:21.871376 18366 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:21.873107 12085 consensus_peers.cc:597] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.873798 12085 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.875061 12085 consensus_peers.cc:597] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.875761 12085 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.878798 12085 leader_election.cc:304] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.870875 18346 raft_consensus.cc:3060] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.879655 18374 raft_consensus.cc:2749] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:21.881443 12085 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.881897 12085 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.882655 18374 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:21.883805 12085 leader_election.cc:336] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.884264 12085 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.885058 18374 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:18:21.886164 12085 leader_election.cc:336] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.886633 12085 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.887430 18374 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.886384 11294 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 10 to 11, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 11 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
W20260709 13:18:21.888537 12085 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.889041 12085 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.889823 18374 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:21.891315 12085 leader_election.cc:336] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.891760 12085 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.891894 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.892627 18374 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.893688 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.893983 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.894516 12113 raft_consensus.cc:2393] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34fe1737ebb840fb9746980f649cc0a6 in current term 10: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
I20260709 13:18:21.895496 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.896751 12085 leader_election.cc:336] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.896962 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.897254 12085 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.898007 18374 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:21.899320 12085 leader_election.cc:336] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.899798 12085 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.899863 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.900599 18374 raft_consensus.cc:2749] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:21.901619 18344 raft_consensus.cc:493] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
W20260709 13:18:21.901753 12085 leader_election.cc:336] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.902283 12085 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.902184 18344 raft_consensus.cc:515] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:21.903097 18374 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:18:21.904209 12085 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.905093 12085 leader_election.cc:336] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.905390 18344 leader_election.cc:290] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:21.905459 12085 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.907094 18374 raft_consensus.cc:2749] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.907436 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
W20260709 13:18:21.908321 12085 leader_election.cc:336] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.908772 12085 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.909500 18374 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.857641 10252 raft_consensus.cc:2243] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:21.910574 12085 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.911054 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.911404 10252 raft_consensus.cc:2272] T cfd1f78aa1d843c28021b808cf5fd960 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.911846 18374 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.914119 18373 raft_consensus.cc:2749] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.915831 12123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b150719d534f439aa33eb70af414a5d0" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.916075 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13474bcfb1f1443f82e90db3445bc880" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.917495 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bea11400ce497283dceb7d21b6e728" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.917701 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af340b26264692af5490466dd69b78" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.918649 11442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.3.1:37459, user_credentials={real_user=slave}} blocked reactor thread for 51953.5us
W20260709 13:18:21.919358 11442 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.919371 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f615bf163b9483d8209eb6c8fedca15" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
W20260709 13:18:21.920359 11442 consensus_peers.cc:597] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.920923 11442 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.921130 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b66bfe782a6d489fa35dc079b8c4d0a5" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.921286 11442 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.922712 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8eacd98a9dc4961a9fac3d2ae9dbd02" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.924551 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10b16798ae24794b186d5daa625c6d1" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.926038 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dba44ee610b54495a5f1cfea0a876524" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.927295 12115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff56d6db2ff44eb19771b9166c042077" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.928668 12119 raft_consensus.cc:1240] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:21.929440 12116 raft_consensus.cc:1240] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Rejecting Update request from peer 147201d22a694ca59f6e014f083a0892 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:21.932062 12118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e67029edeb04a569eb2c35676ef78da" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:21.927577 12113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abd40007d144575bd9e785e609a62df" candidate_uuid: "147201d22a694ca59f6e014f083a0892" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:21.936690 12113 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:18:21.937168 12113 pending_rounds.cc:85] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 10
I20260709 13:18:21.937461 12113 pending_rounds.cc:107] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:18:21.937814 12113 raft_consensus.cc:2889] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:21.939605 11366 leader_election.cc:304] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.940874 18355 raft_consensus.cc:2749] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:21.941725 10252 tablet_replica.cc:333] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.942727 11366 leader_election.cc:304] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.943014 10252 raft_consensus.cc:2243] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:21.943562 11366 raft_consensus.cc:2686] T 13474bcfb1f1443f82e90db3445bc880 P 147201d22a694ca59f6e014f083a0892: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:18:21.943815 10252 raft_consensus.cc:2272] T 6bea8de48c2942d99a1477b5b427be03 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.944906 11366 leader_election.cc:304] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.945752 18355 raft_consensus.cc:2749] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.947105 18289 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.948524 11441 leader_election.cc:304] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.949261 18289 raft_consensus.cc:2749] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:21.949538 12085 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.950003 12085 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:21.950284 11442 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.950749 18354 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:18:21.951360 11442 leader_election.cc:336] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.951951 12085 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.955456 11442 leader_election.cc:304] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.955503 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:21.956399 18315 raft_consensus.cc:2749] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.956488 18354 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:21.957769 11442 consensus_peers.cc:597] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.960114 10252 tablet_replica.cc:333] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
W20260709 13:18:21.960676 11442 leader_election.cc:336] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.961161 10252 raft_consensus.cc:2243] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.961133 11442 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.961932 10252 raft_consensus.cc:2272] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.962387 18312 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:18:21.963740 11442 consensus_peers.cc:597] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.970323 11442 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.970791 11442 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.971531 18373 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:21.975521 11442 consensus_peers.cc:597] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.976142 11442 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.946645 11366 leader_election.cc:304] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.977080 18346 raft_consensus.cc:2749] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:18:21.977139 11442 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:21.978154 11442 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.978533 11366 leader_election.cc:304] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.979068 11442 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.979365 18346 raft_consensus.cc:2749] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.980264 11366 leader_election.cc:304] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.989012 11442 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.989509 11442 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.989574 11366 leader_election.cc:304] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.990660 11442 leader_election.cc:336] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.990866 11366 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.990947 18355 raft_consensus.cc:2749] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.992128 11442 leader_election.cc:304] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:21.991699 18346 raft_consensus.cc:2749] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:21.992904 18355 raft_consensus.cc:2749] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:21.993978 18289 raft_consensus.cc:2749] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.993641 18312 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:21.965144 10252 tablet_replica.cc:333] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:21.992865 11366 leader_election.cc:304] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
W20260709 13:18:21.995169 11442 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:21.995626 10252 raft_consensus.cc:2243] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.997556 10252 raft_consensus.cc:2272] T 59af340b26264692af5490466dd69b78 P 147201d22a694ca59f6e014f083a0892 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.997687 18346 raft_consensus.cc:2749] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:21.997649 11442 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:21.998386 18310 consensus_queue.cc:1059] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:21.999590 18310 raft_consensus.cc:3055] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.999270 11366 leader_election.cc:304] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.000051 18310 raft_consensus.cc:740] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
W20260709 13:18:22.000248 11366 raft_consensus.cc:2686] T ff56d6db2ff44eb19771b9166c042077 P 147201d22a694ca59f6e014f083a0892: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:18:22.000872 18310 consensus_queue.cc:260] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.001416 18344 consensus_queue.cc:1059] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:22.002354 18310 raft_consensus.cc:3060] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:22.002489 10252 tablet_replica.cc:333] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.002971 11366 leader_election.cc:304] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.003561 10252 raft_consensus.cc:2243] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.003775 18346 raft_consensus.cc:2749] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:22.004272 10252 raft_consensus.cc:2272] T 38b39fd56a8d45f7b56a9ac114b66d2b P 147201d22a694ca59f6e014f083a0892 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.004220 11366 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 147201d22a694ca59f6e014f083a0892; no voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.005184 18346 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:22.005997 18344 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:22.007650 18355 raft_consensus.cc:3055] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:22.008098 18355 raft_consensus.cc:740] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 147201d22a694ca59f6e014f083a0892, State: Running, Role: LEADER
I20260709 13:18:22.008997 18355 consensus_queue.cc:260] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.010401 18312 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:22.010481 18355 raft_consensus.cc:3060] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:22.011574 11442 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.015486 11442 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.019181 18312 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:22.027616 11442 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.028079 10252 tablet_replica.cc:333] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.032968 10252 raft_consensus.cc:2243] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.028156 11442 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.033586 10252 raft_consensus.cc:2272] T 74bea11400ce497283dceb7d21b6e728 P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.033869 18312 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:22.034795 11442 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.035588 11442 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.036469 18312 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:22.037920 11442 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.038426 11442 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.039018 10252 tablet_replica.cc:333] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.039243 18312 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:18:22.039904 10252 raft_consensus.cc:2243] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.040542 10252 raft_consensus.cc:2272] T b10b16798ae24794b186d5daa625c6d1 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:22.042336 11442 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.042860 11442 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.043443 10252 tablet_replica.cc:333] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.043686 18312 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:18:22.044268 10252 raft_consensus.cc:2243] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.045140 10252 pending_rounds.cc:70] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Trying to abort 2 pending ops.
W20260709 13:18:22.045154 11442 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.051164 11442 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.045398 10252 pending_rounds.cc:77] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Aborting op as it isn't in flight: id { term: 24 index: 18 } timestamp: 7305638304203780096 op_type: NO_OP noop_request { }
I20260709 13:18:22.052721 10252 raft_consensus.cc:2889] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:22.053086 10252 pending_rounds.cc:77] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892: Aborting op as it isn't in flight: id { term: 22 index: 17 } timestamp: 7305638286695649280 op_type: NO_OP noop_request { }
I20260709 13:18:22.053586 10252 raft_consensus.cc:2889] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:22.053911 10252 raft_consensus.cc:2272] T 5e8ed42aaa39460bbea1f8deeccd0adc P 147201d22a694ca59f6e014f083a0892 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.054253 18289 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:22.056888 10252 tablet_replica.cc:333] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.057752 10252 raft_consensus.cc:2243] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.058276 10252 raft_consensus.cc:2272] T 71ff2b5fd08d4882a5355cf7758991cf P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.061012 10252 tablet_replica.cc:333] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.061877 10252 raft_consensus.cc:2243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.062932 10252 raft_consensus.cc:2272] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.065689 10252 tablet_replica.cc:333] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.066365 10252 raft_consensus.cc:2243] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.066906 10252 raft_consensus.cc:2272] T 79d2059876e34ff89bf967e96ab7bf09 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.069337 10252 tablet_replica.cc:333] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.070075 10252 raft_consensus.cc:2243] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.070576 10252 raft_consensus.cc:2272] T 7abd40007d144575bd9e785e609a62df P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.073268 10252 tablet_replica.cc:333] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.074007 10252 raft_consensus.cc:2243] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.074506 10252 raft_consensus.cc:2272] T c3fd80658464474dbe463b1943becc75 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.077072 10252 tablet_replica.cc:333] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.077762 10252 raft_consensus.cc:2243] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.078763 10252 raft_consensus.cc:2272] T 1c4eb2d16aa247998d0927eb34dedfa8 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.081076 10252 tablet_replica.cc:333] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.081741 10252 raft_consensus.cc:2243] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.082556 10252 raft_consensus.cc:2272] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.085116 10252 tablet_replica.cc:333] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.085807 10252 raft_consensus.cc:2243] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.085700 18333 consensus_queue.cc:1059] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:22.086426 10252 raft_consensus.cc:2272] T 255638a15529487e8ade55aaaa8455d5 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.086869 18333 raft_consensus.cc:3055] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.087327 18333 raft_consensus.cc:740] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:22.089241 10252 tablet_replica.cc:333] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.088402 18333 consensus_queue.cc:260] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.090047 10252 raft_consensus.cc:2243] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.090581 10252 raft_consensus.cc:2272] T 1f946b7973fe482681135635493ebd48 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.092164 18333 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.093197 10252 tablet_replica.cc:333] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.093997 10252 raft_consensus.cc:2243] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.094509 10252 raft_consensus.cc:2272] T 3c4de37bdc334587a90c03b40bfd6c5f P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.097084 10252 tablet_replica.cc:333] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.097833 10252 raft_consensus.cc:2243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.098320 10252 raft_consensus.cc:2272] T e8e0fc54e8b645deb82c6013c5d5c7ea P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.101321 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43c1c5858a0f4571a21e2d097910aa44" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.101768 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.102072 10252 tablet_replica.cc:333] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.103057 10252 raft_consensus.cc:2243] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.103611 10252 raft_consensus.cc:2272] T 4f615bf163b9483d8209eb6c8fedca15 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.103828 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.105897 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b606c6942f42cf914bcabc7e2bc502" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.106410 10252 tablet_replica.cc:333] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.107363 10252 raft_consensus.cc:2243] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.108083 10252 raft_consensus.cc:2272] T 6d02abb4f52449679a2415346f354340 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.110852 10252 tablet_replica.cc:333] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.111676 10252 raft_consensus.cc:2243] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.112242 10252 raft_consensus.cc:2272] T 6eba703fc47843af924d4e45f12db16b P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.113528 11545 raft_consensus.cc:2468] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 10.
I20260709 13:18:22.115819 18333 consensus_queue.cc:1059] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:22.117374 18300 raft_consensus.cc:3055] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.117743 18300 raft_consensus.cc:740] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:22.118511 18300 consensus_queue.cc:260] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.119905 18300 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.126382 11442 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.128651 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2dfc6e65bda4ed2a39f4bce45ec85e6" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.129326 10252 tablet_replica.cc:333] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.129307 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "34fe1737ebb840fb9746980f649cc0a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.130592 10252 raft_consensus.cc:2243] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.131381 10252 raft_consensus.cc:2272] T dba44ee610b54495a5f1cfea0a876524 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.134558 10252 tablet_replica.cc:333] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.135637 11544 raft_consensus.cc:2468] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:22.138893 18373 raft_consensus.cc:2749] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:22.140133 10252 raft_consensus.cc:2243] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.141284 10252 raft_consensus.cc:2272] T a3205d9e6dce4aad891fa90943f21879 P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.142791 11543 raft_consensus.cc:2468] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fe1737ebb840fb9746980f649cc0a6 in term 14.
I20260709 13:18:22.144097 10252 tablet_replica.cc:333] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.144912 10252 raft_consensus.cc:2243] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.145555 10252 raft_consensus.cc:2272] T b61ac66d97f84c1186c6e2b1d79477b2 P 147201d22a694ca59f6e014f083a0892 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.139430 11442 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.148984 18373 raft_consensus.cc:2749] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:22.151113 11442 leader_election.cc:304] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.148212 10252 tablet_replica.cc:333] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.151993 18373 raft_consensus.cc:2804] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:22.152231 10252 raft_consensus.cc:2243] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.153313 10252 raft_consensus.cc:2272] T ffe5740695024e83afc5069cfb41118c P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.152500 18373 raft_consensus.cc:697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:22.156217 18373 consensus_queue.cc:237] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.160921 11442 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.162192 10252 tablet_replica.cc:333] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.163122 10252 raft_consensus.cc:2243] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.163372 18289 raft_consensus.cc:2749] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:22.163604 10252 raft_consensus.cc:2272] T b5de422ef4884189bea49cec7b7c2ec1 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.165752 11442 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.166553 18289 raft_consensus.cc:2804] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:22.166879 10252 tablet_replica.cc:333] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.167060 18289 raft_consensus.cc:697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:22.168056 18289 consensus_queue.cc:237] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.167847 10252 raft_consensus.cc:2243] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.166923 11442 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b; no voters: b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:22.172092 10252 raft_consensus.cc:2272] T 4e67029edeb04a569eb2c35676ef78da P 147201d22a694ca59f6e014f083a0892 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.171375 18316 raft_consensus.cc:2804] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:22.174891 18316 raft_consensus.cc:697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Becoming Leader. State: Replica: 34fe1737ebb840fb9746980f649cc0a6, State: Running, Role: LEADER
I20260709 13:18:22.175774 11471 raft_consensus.cc:1240] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:22.175930 18316 consensus_queue.cc:237] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:22.178998 11442 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.179430 11442 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.179534 11472 raft_consensus.cc:1240] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:22.180119 18294 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:18:22.180416 11442 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:22.180815 11477 raft_consensus.cc:1240] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Rejecting Update request from peer 9e9c2603c0bd4d25b713649b3890c46b for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:22.180816 11442 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:22.182466 18373 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:22.188138 11294 catalog_manager.cc:5697] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 9 to 10, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 10 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.192294 18300 consensus_queue.cc:1059] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 16.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:18:22.193302 18366 raft_consensus.cc:3055] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:22.193692 18366 raft_consensus.cc:740] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:22.193912 18334 consensus_queue.cc:1059] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:22.194948 18334 raft_consensus.cc:3055] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.194568 18366 consensus_queue.cc:260] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.195374 18334 raft_consensus.cc:740] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:22.196131 18366 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:22.196290 18334 consensus_queue.cc:260] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.197852 18334 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.188200 10252 tablet_replica.cc:333] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.201537 10252 raft_consensus.cc:2243] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.202081 10252 raft_consensus.cc:2272] T b66bfe782a6d489fa35dc079b8c4d0a5 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.204257 18333 consensus_queue.cc:1059] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:22.205294 18333 raft_consensus.cc:3055] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:22.205597 10252 tablet_replica.cc:333] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.205653 18333 raft_consensus.cc:740] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:22.206574 10252 raft_consensus.cc:2243] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.206691 18333 consensus_queue.cc:260] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.207597 10252 raft_consensus.cc:2272] T b150719d534f439aa33eb70af414a5d0 P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.208092 18333 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:22.210404 10252 tablet_replica.cc:333] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.211211 10252 raft_consensus.cc:2243] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.213217 10252 raft_consensus.cc:2272] T f52a477150c345c8ba80211d1a49b41c P 147201d22a694ca59f6e014f083a0892 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.215950 10252 tablet_replica.cc:333] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.216719 10252 raft_consensus.cc:2243] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.217265 10252 raft_consensus.cc:2272] T 930d965be7eb49448fa23456beb9925a P 147201d22a694ca59f6e014f083a0892 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.222424 12113 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:18:22.222786 12113 pending_rounds.cc:85] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting all ops after (but not including) 12
I20260709 13:18:22.223016 12113 pending_rounds.cc:107] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:18:22.223245 12113 raft_consensus.cc:2889] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.224493 11543 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer 147201d22a694ca59f6e014f083a0892: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:22.224588 18310 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:22.224695 10252 tablet_replica.cc:333] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.226256 18310 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:22.225941 11294 catalog_manager.cc:5697] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.226843 10252 raft_consensus.cc:2243] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.227990 10252 raft_consensus.cc:2272] T 840a1b524ac54acdb12289fa1aafbd17 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.227692 11294 catalog_manager.cc:5697] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2). New cstate: current_term: 14 leader_uuid: "34fe1737ebb840fb9746980f649cc0a6" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.233203 10252 tablet_replica.cc:333] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.234546 10252 raft_consensus.cc:2243] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.235402 10252 raft_consensus.cc:2272] T e106bf87136140b4bebad713d83d02fe P 147201d22a694ca59f6e014f083a0892 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.238541 10252 tablet_replica.cc:333] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.239768 10252 raft_consensus.cc:2243] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.240360 10252 raft_consensus.cc:2272] T 5b22c9e294a846159d4831b92f692943 P 147201d22a694ca59f6e014f083a0892 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.243234 10252 tablet_replica.cc:333] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.244115 10252 raft_consensus.cc:2243] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.244683 10252 raft_consensus.cc:2272] T aa999fcc28454bc88f93770d07ff72e2 P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.247380 10252 tablet_replica.cc:333] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.248140 10252 raft_consensus.cc:2243] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.252571 10252 raft_consensus.cc:2272] T 48129c3807a242fca6439f00aefd9b22 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.255585 10252 tablet_replica.cc:333] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.258056 11293 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 16 to 17, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 147201d22a694ca59f6e014f083a0892 (127.10.3.1). New cstate: current_term: 17 leader_uuid: "147201d22a694ca59f6e014f083a0892" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } } }
I20260709 13:18:22.264272 10252 raft_consensus.cc:2243] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.264827 10252 raft_consensus.cc:2272] T 44539667cbc04657b801ff659dee5bde P 147201d22a694ca59f6e014f083a0892 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.266808 10252 tablet_replica.cc:333] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.267475 10252 raft_consensus.cc:2243] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.268539 10252 raft_consensus.cc:2272] T eeb668f66ce344e9acdf8834c2b1d3a3 P 147201d22a694ca59f6e014f083a0892 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.274927 10252 tablet_replica.cc:333] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892: stopping tablet replica
I20260709 13:18:22.275679 10252 raft_consensus.cc:2243] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.276180 10252 raft_consensus.cc:2272] T 08ebf4312af74188830a07342d0dc119 P 147201d22a694ca59f6e014f083a0892 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.636168 11544 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 13:18:22.637568 18315 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:22.646099 12113 raft_consensus.cc:3060] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.659773 12113 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:22.668968 12118 raft_consensus.cc:1275] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 13:18:22.674256 18316 consensus_queue.cc:1048] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:22.675900 18316 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:22.700109 11544 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:22.702320 18294 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:22.703810 12116 raft_consensus.cc:3060] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:22.740093 12116 raft_consensus.cc:1275] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:22.777428 11543 raft_consensus.cc:1275] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer 34fe1737ebb840fb9746980f649cc0a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:22.807766 18315 consensus_queue.cc:1048] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:22.825947 18315 consensus_queue.cc:1048] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:22.911688 18405 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:22.912271 18405 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.929553 18405 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:22.931993 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:22.932963 11477 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 18: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:22.931941 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:22.934412 11543 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:22.935746 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:22.937264 18407 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:22.937747 18407 raft_consensus.cc:493] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:22.938170 18407 raft_consensus.cc:3060] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:22.944834 18407 raft_consensus.cc:515] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.946617 18407 leader_election.cc:290] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:22.947342 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:22.947649 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "902c85625c4647d481fa8ef07be336f4" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:22.948165 11477 raft_consensus.cc:2393] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b3abf4b5ee724203ad1226f5c46bfe46 in current term 18: Already voted for candidate 34fe1737ebb840fb9746980f649cc0a6 in this term.
I20260709 13:18:22.954790 11543 raft_consensus.cc:2468] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 18.
I20260709 13:18:22.956357 12085 leader_election.cc:304] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:22.958006 18407 raft_consensus.cc:2804] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:22.958659 18407 raft_consensus.cc:697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:22.959643 18407 consensus_queue.cc:237] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:22.973645 11293 catalog_manager.cc:5697] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 17 to 18, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 18 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.032977 18407 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.034466 18407 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.037050 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:23.037639 11477 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:18:23.039108 12083 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.039804 18405 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:23.040176 18405 raft_consensus.cc:493] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.040496 18405 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:23.043370 18406 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.043820 18407 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.043910 18406 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.044760 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.045348 11543 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:18:23.046243 18406 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.047538 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.047745 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6" is_pre_election: true
I20260709 13:18:23.048385 11477 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 10.
I20260709 13:18:23.049273 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:23.049919 18406 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:23.050311 18406 raft_consensus.cc:493] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.050616 18406 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.051131 18405 raft_consensus.cc:515] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.058974 18406 raft_consensus.cc:515] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.060878 18405 leader_election.cc:290] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.061426 18406 leader_election.cc:290] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.065853 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.066648 11543 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:23.068943 11472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:23.069516 11472 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.065903 11477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a11e4197bc8843c69871c3f11dc98474" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34fe1737ebb840fb9746980f649cc0a6"
I20260709 13:18:23.072220 11477 raft_consensus.cc:3060] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:23.073361 11543 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:18:23.074774 10252 tablet_server.cc:196] TabletServer@127.10.3.1:0 shutdown complete.
I20260709 13:18:23.076683 12085 leader_election.cc:304] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.077812 11472 raft_consensus.cc:2468] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:18:23.078271 18405 raft_consensus.cc:2804] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:23.083989 12083 leader_election.cc:304] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34fe1737ebb840fb9746980f649cc0a6, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.084893 18407 raft_consensus.cc:2804] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:23.085431 18407 raft_consensus.cc:697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.086326 18407 consensus_queue.cc:237] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.083029 18405 raft_consensus.cc:697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.088382 18405 consensus_queue.cc:237] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.079339 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778170f25cb24e08b57cbbc852e8a007" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.080464 11477 raft_consensus.cc:2468] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 13.
I20260709 13:18:23.103147 11293 catalog_manager.cc:5697] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.121361 11293 catalog_manager.cc:5697] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.186740 10252 tablet_server.cc:179] TabletServer@127.10.3.2:0 shutting down...
I20260709 13:18:23.232573 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:23.233520 10252 tablet_replica.cc:333] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.234555 10252 raft_consensus.cc:2243] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.235201 10252 raft_consensus.cc:2272] T 95e3d59427ee46f989532eb3cdc311cc P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.237766 10252 tablet_replica.cc:333] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.238312 10252 raft_consensus.cc:2243] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.239130 10252 raft_consensus.cc:2272] T 66cb4598d33043fc8eb235647a242814 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.240885 10252 tablet_replica.cc:333] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.241410 10252 raft_consensus.cc:2243] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.241787 10252 raft_consensus.cc:2272] T b61ac66d97f84c1186c6e2b1d79477b2 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.243510 10252 tablet_replica.cc:333] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.243985 10252 raft_consensus.cc:2243] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.244697 10252 raft_consensus.cc:2272] T 058fa5ec3edb42c796103c7361c1a8e1 P 34fe1737ebb840fb9746980f649cc0a6 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.246337 10252 tablet_replica.cc:333] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.246848 10252 raft_consensus.cc:2243] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.247357 10252 raft_consensus.cc:2272] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.249162 10252 tablet_replica.cc:333] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.249694 10252 raft_consensus.cc:2243] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.250113 10252 raft_consensus.cc:2272] T b5de422ef4884189bea49cec7b7c2ec1 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.251855 10252 tablet_replica.cc:333] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.252442 10252 raft_consensus.cc:2243] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.253131 10252 raft_consensus.cc:2272] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.255834 10252 tablet_replica.cc:333] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.256428 10252 raft_consensus.cc:2243] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.257145 10252 raft_consensus.cc:2272] T cfd1f78aa1d843c28021b808cf5fd960 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.259189 10252 tablet_replica.cc:333] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.259713 10252 raft_consensus.cc:2243] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.260115 10252 raft_consensus.cc:2272] T fe707c46816d4d84bf4ead0d04de2b26 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.261818 10252 tablet_replica.cc:333] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.262320 10252 raft_consensus.cc:2243] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.262745 10252 raft_consensus.cc:2272] T 6bea8de48c2942d99a1477b5b427be03 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.264714 10252 tablet_replica.cc:333] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.265336 10252 raft_consensus.cc:2243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.265796 10252 raft_consensus.cc:2272] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.267601 10252 tablet_replica.cc:333] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.268088 10252 raft_consensus.cc:2243] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.268815 10252 raft_consensus.cc:2272] T 7abd40007d144575bd9e785e609a62df P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.270951 10252 tablet_replica.cc:333] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.271799 10252 raft_consensus.cc:2243] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.272740 10252 raft_consensus.cc:2272] T c3fd80658464474dbe463b1943becc75 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.274611 10252 tablet_replica.cc:333] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.275252 18407 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.275337 10252 raft_consensus.cc:2243] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.275789 18407 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.276304 10252 raft_consensus.cc:2272] T a6454d815ae44cdda2fe8d013fca9b4b P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.277805 18407 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.278780 10252 tablet_replica.cc:333] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.279644 10252 raft_consensus.cc:2243] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.279527 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.280167 11543 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
I20260709 13:18:23.280652 10252 raft_consensus.cc:2272] T 71ff2b5fd08d4882a5355cf7758991cf P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:23.282073 12083 leader_election.cc:336] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.283356 10252 tablet_replica.cc:333] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.283936 18366 raft_consensus.cc:493] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.284202 10252 raft_consensus.cc:2243] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.284410 18366 raft_consensus.cc:515] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.285274 10252 raft_consensus.cc:2272] T 56b606c6942f42cf914bcabc7e2bc502 P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:23.285701 12083 consensus_peers.cc:597] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.285230 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.287153 18407 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:23.287828 18407 raft_consensus.cc:493] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.288187 18407 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:23.289718 10252 tablet_replica.cc:333] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.290642 10252 raft_consensus.cc:2243] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.291846 18366 leader_election.cc:290] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.292277 18405 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.292726 18405 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.293771 10252 raft_consensus.cc:2272] T 38b39fd56a8d45f7b56a9ac114b66d2b P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.294744 18405 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.295313 18366 raft_consensus.cc:493] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:18:23.295518 11514 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.295867 18366 raft_consensus.cc:515] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.296535 10252 tablet_replica.cc:333] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.296769 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.297323 10252 raft_consensus.cc:2243] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.297351 11543 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:23.297981 10252 raft_consensus.cc:2272] T 902c85625c4647d481fa8ef07be336f4 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.298555 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.299345 18405 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:23.299711 18405 raft_consensus.cc:493] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.300071 18405 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:23.300767 10252 tablet_replica.cc:333] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.301860 10252 raft_consensus.cc:2243] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.302829 10252 raft_consensus.cc:2272] T 74bea11400ce497283dceb7d21b6e728 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.305398 10252 tablet_replica.cc:333] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
W20260709 13:18:23.305673 11514 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.306156 10252 raft_consensus.cc:2243] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:23.306861 11516 leader_election.cc:336] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.307338 11516 leader_election.cc:304] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.307811 10252 raft_consensus.cc:2272] T 1c4eb2d16aa247998d0927eb34dedfa8 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.308291 18333 raft_consensus.cc:2749] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:23.310527 10252 tablet_replica.cc:333] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.311411 10252 raft_consensus.cc:2243] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:23.311484 12083 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.298568 18366 leader_election.cc:290] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.312115 10252 raft_consensus.cc:2272] T 255638a15529487e8ade55aaaa8455d5 P 34fe1737ebb840fb9746980f649cc0a6 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:23.312922 11514 consensus_peers.cc:597] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.313146 18407 raft_consensus.cc:515] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:23.313848 11516 leader_election.cc:336] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.314285 11516 leader_election.cc:304] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.314844 10252 tablet_replica.cc:333] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.315440 18366 raft_consensus.cc:2749] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:23.315721 10252 raft_consensus.cc:2243] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.316282 10252 raft_consensus.cc:2272] T 1f946b7973fe482681135635493ebd48 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.310078 18405 raft_consensus.cc:515] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.316751 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6454d815ae44cdda2fe8d013fca9b4b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.317309 11543 raft_consensus.cc:3060] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:18:23.318380 10252 tablet_replica.cc:333] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.318485 18405 leader_election.cc:290] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.319164 18407 leader_election.cc:290] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.319304 10252 raft_consensus.cc:2243] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.319459 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12c14d0fd7ef4fbf899a4b3eebe33e5b" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.320060 10252 raft_consensus.cc:2272] T a11e4197bc8843c69871c3f11dc98474 P 34fe1737ebb840fb9746980f649cc0a6 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.320081 11544 raft_consensus.cc:3060] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:23.322788 10252 tablet_replica.cc:333] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.323575 10252 raft_consensus.cc:2243] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.324175 10252 raft_consensus.cc:2272] T 25be57d1c24d4a43859ce0b5848adc09 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.323083 11543 raft_consensus.cc:2468] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 16.
I20260709 13:18:23.326951 10252 tablet_replica.cc:333] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.327773 10252 raft_consensus.cc:2243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.328377 10252 raft_consensus.cc:2272] T e8e0fc54e8b645deb82c6013c5d5c7ea P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:23.328577 12083 leader_election.cc:336] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:23.329392 12083 leader_election.cc:336] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.330133 12085 leader_election.cc:304] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.330856 10252 tablet_replica.cc:333] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.330825 18407 raft_consensus.cc:2804] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:23.331148 18366 raft_consensus.cc:493] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.331666 18407 raft_consensus.cc:697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.331874 10252 raft_consensus.cc:2243] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.331756 11544 raft_consensus.cc:2468] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:23.333014 10252 raft_consensus.cc:2272] T 4f615bf163b9483d8209eb6c8fedca15 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.332734 18366 raft_consensus.cc:515] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.332701 18407 consensus_queue.cc:237] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.333165 12085 leader_election.cc:304] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.334177 18405 raft_consensus.cc:2804] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:23.334908 18405 raft_consensus.cc:697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.335923 10252 tablet_replica.cc:333] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.335786 18405 consensus_queue.cc:237] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.336740 10252 raft_consensus.cc:2243] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.337433 10252 raft_consensus.cc:2272] T ffe5740695024e83afc5069cfb41118c P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:23.338907 11514 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.339993 10252 tablet_replica.cc:333] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.339951 18366 leader_election.cc:290] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.340956 10252 raft_consensus.cc:2243] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.341571 10252 raft_consensus.cc:2272] T a3205d9e6dce4aad891fa90943f21879 P 34fe1737ebb840fb9746980f649cc0a6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.344101 10252 tablet_replica.cc:333] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.344802 10252 raft_consensus.cc:2243] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.345784 10252 raft_consensus.cc:2272] T dba44ee610b54495a5f1cfea0a876524 P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.348358 10252 tablet_replica.cc:333] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.349099 10252 raft_consensus.cc:2243] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.348682 11293 catalog_manager.cc:5697] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.349658 10252 raft_consensus.cc:2272] T aa999fcc28454bc88f93770d07ff72e2 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.352227 10252 tablet_replica.cc:333] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.352937 10252 raft_consensus.cc:2243] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.353848 10252 raft_consensus.cc:2272] T 48129c3807a242fca6439f00aefd9b22 P 34fe1737ebb840fb9746980f649cc0a6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.356559 10252 tablet_replica.cc:333] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.357251 10252 raft_consensus.cc:2243] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.357798 10252 raft_consensus.cc:2272] T 44539667cbc04657b801ff659dee5bde P 34fe1737ebb840fb9746980f649cc0a6 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.360320 10252 tablet_replica.cc:333] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.360967 10252 raft_consensus.cc:2243] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.361451 10252 raft_consensus.cc:2272] T b2eeda82ae3942d6bf0db873fa350bdc P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.363999 10252 tablet_replica.cc:333] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.364751 10252 raft_consensus.cc:2243] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
W20260709 13:18:23.365291 11516 leader_election.cc:336] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.365787 10252 raft_consensus.cc:2272] T 5e9e663b46c5417b9a0e42fdc7b70969 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.365763 11516 leader_election.cc:304] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.368117 18366 raft_consensus.cc:2749] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:23.368321 10252 tablet_replica.cc:333] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.369202 10252 raft_consensus.cc:2243] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.369911 10252 raft_consensus.cc:2272] T e106bf87136140b4bebad713d83d02fe P 34fe1737ebb840fb9746980f649cc0a6 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.373121 10252 tablet_replica.cc:333] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.373843 10252 raft_consensus.cc:2243] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.374351 10252 raft_consensus.cc:2272] T 5b22c9e294a846159d4831b92f692943 P 34fe1737ebb840fb9746980f649cc0a6 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.376897 10252 tablet_replica.cc:333] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.377570 10252 raft_consensus.cc:2243] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.377265 11293 catalog_manager.cc:5697] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.378296 10252 raft_consensus.cc:2272] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.380836 10252 tablet_replica.cc:333] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.381531 10252 raft_consensus.cc:2243] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.382242 10252 raft_consensus.cc:2272] T 6eba703fc47843af924d4e45f12db16b P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.384167 10252 tablet_replica.cc:333] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.384687 10252 raft_consensus.cc:2243] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.385123 10252 raft_consensus.cc:2272] T 778170f25cb24e08b57cbbc852e8a007 P 34fe1737ebb840fb9746980f649cc0a6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.387354 10252 tablet_replica.cc:333] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.387856 10252 raft_consensus.cc:2243] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.388607 10252 raft_consensus.cc:2272] T 9b37b6683af04823b0814ba56c4ea803 P 34fe1737ebb840fb9746980f649cc0a6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.390383 10252 tablet_replica.cc:333] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.390872 10252 raft_consensus.cc:2243] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.391777 10252 raft_consensus.cc:2272] T f52a477150c345c8ba80211d1a49b41c P 34fe1737ebb840fb9746980f649cc0a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.393529 10252 tablet_replica.cc:333] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.394026 10252 raft_consensus.cc:2243] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.394735 10252 raft_consensus.cc:2272] T 43c1c5858a0f4571a21e2d097910aa44 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.396878 10252 tablet_replica.cc:333] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.397413 10252 raft_consensus.cc:2243] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.397820 10252 raft_consensus.cc:2272] T 5e8ed42aaa39460bbea1f8deeccd0adc P 34fe1737ebb840fb9746980f649cc0a6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.400007 10252 tablet_replica.cc:333] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.400569 10252 raft_consensus.cc:2243] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:18:23.401562 10252 raft_consensus.cc:2272] T b10b16798ae24794b186d5daa625c6d1 P 34fe1737ebb840fb9746980f649cc0a6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.404124 10252 tablet_replica.cc:333] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6: stopping tablet replica
I20260709 13:18:23.404531 10252 raft_consensus.cc:2243] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:23.404906 10252 raft_consensus.cc:2272] T 08ebf4312af74188830a07342d0dc119 P 34fe1737ebb840fb9746980f649cc0a6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:23.405411 18366 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.406023 18366 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.409224 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:23.408237 18366 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
W20260709 13:18:23.412348 11516 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.412766 11516 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:23.413478 18366 raft_consensus.cc:2749] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.430776 18405 raft_consensus.cc:493] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.431365 18405 raft_consensus.cc:515] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.432531 18406 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:23.433310 18405 leader_election.cc:290] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.433053 18406 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.435266 18406 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.436981 11544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.437690 11544 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
W20260709 13:18:23.439010 12083 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.440231 18406 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.440727 18406 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:23.442296 12085 leader_election.cc:336] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.442838 12085 leader_election.cc:304] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.442935 18406 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.443827 18405 raft_consensus.cc:2749] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.444492 18405 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:18:23.444617 12083 leader_election.cc:336] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.445067 18405 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.447358 18405 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.447430 12083 leader_election.cc:336] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.448597 12085 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.449359 18405 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:23.449805 18405 raft_consensus.cc:493] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:23.450207 18405 raft_consensus.cc:3060] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:23.451217 12083 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.459189 11544 raft_consensus.cc:1275] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:23.463397 18406 consensus_queue.cc:1048] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:18:23.463460 12085 leader_election.cc:336] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.464342 12085 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.465068 18406 raft_consensus.cc:2749] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:18:23.465310 12085 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.465777 12085 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.466476 18406 raft_consensus.cc:2749] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:23.468815 18366 raft_consensus.cc:493] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.469451 18366 raft_consensus.cc:515] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.473266 18406 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.476922 18366 leader_election.cc:290] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.473824 18406 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.479652 18406 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.479928 11514 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.481602 18405 raft_consensus.cc:515] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:23.483103 12083 consensus_peers.cc:597] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.483866 18405 leader_election.cc:290] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:23.485497 11516 leader_election.cc:336] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.485970 11516 leader_election.cc:304] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.487195 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a6ebf03c40d4f1fa404e0fe9f8f9fc5" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.490347 18366 raft_consensus.cc:2749] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:23.493307 12083 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:18:23.494412 12083 leader_election.cc:336] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:23.500186 11543 raft_consensus.cc:2468] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 14.
W20260709 13:18:23.505872 12083 consensus_peers.cc:597] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.509380 12085 leader_election.cc:304] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.510206 18407 raft_consensus.cc:2804] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:23.515057 12085 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.515523 12085 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:23.516381 12083 consensus_peers.cc:597] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.517247 18419 raft_consensus.cc:2749] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.518584 18405 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.522365 18405 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.524452 18405 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.521782 18407 raft_consensus.cc:697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.525941 18407 consensus_queue.cc:237] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.532147 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.532820 11543 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 11.
I20260709 13:18:23.533499 11545 raft_consensus.cc:3060] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.535697 12085 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.536511 18420 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:23.536871 18420 raft_consensus.cc:493] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.537214 18420 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:23.541528 11545 raft_consensus.cc:1275] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:18:23.543980 12085 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.545361 18405 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:23.545549 18420 raft_consensus.cc:515] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.545866 18405 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.547644 18420 leader_election.cc:290] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.547942 18405 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.548200 18420 consensus_queue.cc:1048] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:23.548638 11293 catalog_manager.cc:5697] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 13 to 14, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 14 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:23.553102 12083 leader_election.cc:336] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.554220 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840a1b524ac54acdb12289fa1aafbd17" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:23.554782 11545 raft_consensus.cc:3060] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:23.560075 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.560741 11543 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:23.562285 12085 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.563058 18420 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:23.563500 18420 raft_consensus.cc:493] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9e9c2603c0bd4d25b713649b3890c46b)
I20260709 13:18:23.563900 18420 raft_consensus.cc:3060] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:23.565496 12085 leader_election.cc:336] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.560088 11545 raft_consensus.cc:2468] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 12.
I20260709 13:18:23.571216 18420 raft_consensus.cc:515] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.572420 12085 leader_election.cc:304] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:23.574105 18407 raft_consensus.cc:2804] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:23.574599 18407 raft_consensus.cc:697] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.575829 18407 consensus_queue.cc:237] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.581707 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be57d1c24d4a43859ce0b5848adc09" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
W20260709 13:18:23.582748 11516 consensus_peers.cc:597] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:23.586503 12083 consensus_peers.cc:597] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:23.587047 12083 leader_election.cc:336] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.592034 11545 raft_consensus.cc:2468] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 17.
I20260709 13:18:23.592551 18420 leader_election.cc:290] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.596995 12085 leader_election.cc:304] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.598201 18405 raft_consensus.cc:2804] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:23.598809 18405 raft_consensus.cc:697] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:23.599370 18406 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.599892 18406 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.599818 18405 consensus_queue.cc:237] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.602100 18406 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.597016 12083 consensus_peers.cc:597] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:23.610584 12085 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:23.616080 12083 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.616505 12083 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:23.617380 12083 consensus_peers.cc:597] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.617990 18407 raft_consensus.cc:2749] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:23.623551 11296 catalog_manager.cc:5697] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 11 to 12, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 12 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.625346 11296 catalog_manager.cc:5697] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 16 to 17, leader changed from 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 17 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.629146 18366 raft_consensus.cc:493] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.629685 18366 raft_consensus.cc:515] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.631757 18366 leader_election.cc:290] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.639828 11516 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:23.645581 11514 leader_election.cc:336] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.645939 11514 leader_election.cc:304] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.646521 18366 raft_consensus.cc:2749] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:23.648252 18366 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.648620 18366 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.650574 18366 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:23.651175 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:23.651744 12119 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 19.
I20260709 13:18:23.652947 11515 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.653748 18366 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:23.654124 11514 leader_election.cc:336] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.654155 18366 raft_consensus.cc:493] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.654703 18366 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:23.662066 18366 raft_consensus.cc:515] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.664171 18366 leader_election.cc:290] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:23.665223 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2eeda82ae3942d6bf0db873fa350bdc" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:23.665840 12119 raft_consensus.cc:3060] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:23.669405 11516 consensus_peers.cc:597] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:23.670495 11514 leader_election.cc:336] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.672274 12119 raft_consensus.cc:2468] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:18:23.673316 11515 leader_election.cc:304] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.673965 18333 raft_consensus.cc:2804] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:23.674484 18333 raft_consensus.cc:697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:23.675314 18333 consensus_queue.cc:237] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.680727 18366 raft_consensus.cc:493] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.681509 18366 raft_consensus.cc:515] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:23.684896 11516 consensus_peers.cc:597] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:23.688339 11545 raft_consensus.cc:1275] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:23.689587 18366 leader_election.cc:290] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:23.691948 18407 consensus_queue.cc:1048] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:18:23.692371 11516 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:23.700409 11514 leader_election.cc:336] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.700868 11514 leader_election.cc:304] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.701483 18427 raft_consensus.cc:2749] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:23.701588 11296 catalog_manager.cc:5697] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 18 to 20, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 20 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:23.718875 11516 consensus_peers.cc:597] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:23.731127 18406 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.731762 18406 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.733770 18406 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:23.735951 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:23.736222 18427 raft_consensus.cc:493] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.736764 18427 raft_consensus.cc:515] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
W20260709 13:18:23.739638 12085 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.740113 12085 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:23.740777 18406 raft_consensus.cc:2749] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:18:23.743724 11514 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.744256 18427 leader_election.cc:290] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.746116 11516 leader_election.cc:336] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.746526 11516 leader_election.cc:304] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.747182 18427 raft_consensus.cc:2749] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:18:23.758513 18427 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.759071 18427 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:23.760694 11516 consensus_peers.cc:597] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:23.761140 18427 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:23.762097 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:23.762866 12119 raft_consensus.cc:2393] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e9c2603c0bd4d25b713649b3890c46b in current term 20: Already voted for candidate b3abf4b5ee724203ad1226f5c46bfe46 in this term.
W20260709 13:18:23.764791 11516 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.765195 11516 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, b3abf4b5ee724203ad1226f5c46bfe46
I20260709 13:18:23.765880 18427 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:18:23.769493 18333 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:23.770049 18333 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.771785 18427 raft_consensus.cc:2749] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:18:23.772074 18333 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:23.773600 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:23.774237 12119 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 17.
I20260709 13:18:23.775740 11515 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:23.777282 18427 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:23.777663 18427 raft_consensus.cc:493] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
W20260709 13:18:23.778005 11516 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.778009 18427 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:23.782688 11516 consensus_peers.cc:597] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:23.785620 18427 raft_consensus.cc:515] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.787859 18427 leader_election.cc:290] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:23.789058 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb668f66ce344e9acdf8834c2b1d3a3" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:23.789654 12119 raft_consensus.cc:3060] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:23.791828 11516 leader_election.cc:336] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:23.796079 12119 raft_consensus.cc:2468] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 18.
I20260709 13:18:23.797173 11515 leader_election.cc:304] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:23.797866 18427 raft_consensus.cc:2804] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:23.798341 18427 raft_consensus.cc:697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:23.799181 18427 consensus_queue.cc:237] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:23.809744 11293 catalog_manager.cc:5697] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 17 to 18, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 18 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.811888 11545 raft_consensus.cc:1275] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:23.813234 18406 consensus_queue.cc:1048] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:18:23.830003 12083 consensus_peers.cc:597] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:23.853363 11516 consensus_peers.cc:597] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:23.883498 12085 consensus_peers.cc:597] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:23.885236 11545 raft_consensus.cc:1275] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:23.886917 18405 consensus_queue.cc:1048] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:18:23.905612 12083 consensus_peers.cc:597] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:23.960501 18366 raft_consensus.cc:493] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.961030 18366 raft_consensus.cc:515] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:23.963214 18366 leader_election.cc:290] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:23.967097 11516 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:23.967592 11514 leader_election.cc:336] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:23.968024 11514 leader_election.cc:304] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:23.968688 18366 raft_consensus.cc:2749] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:23.993978 11516 consensus_peers.cc:597] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.000653 18366 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.001179 18366 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.002949 18366 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:24.004079 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" is_pre_election: true
I20260709 13:18:24.004745 12119 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 19.
I20260709 13:18:24.005983 11515 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:24.007120 18366 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:18:24.007525 11514 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:24.007539 18366 raft_consensus.cc:493] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.008777 18366 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:24.012447 12085 consensus_peers.cc:597] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.015139 18366 raft_consensus.cc:515] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.017006 18366 leader_election.cc:290] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Requested vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4:45993)
I20260709 13:18:24.018102 12119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe707c46816d4d84bf4ead0d04de2b26" candidate_uuid: "9e9c2603c0bd4d25b713649b3890c46b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b3abf4b5ee724203ad1226f5c46bfe46"
I20260709 13:18:24.018703 12119 raft_consensus.cc:3060] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:18:24.020920 11514 leader_election.cc:336] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:24.023766 12119 raft_consensus.cc:2468] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e9c2603c0bd4d25b713649b3890c46b in term 20.
I20260709 13:18:24.024734 11515 leader_election.cc:304] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:24.025382 18366 raft_consensus.cc:2804] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:18:24.025874 18366 raft_consensus.cc:697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Becoming Leader. State: Replica: 9e9c2603c0bd4d25b713649b3890c46b, State: Running, Role: LEADER
I20260709 13:18:24.026629 18366 consensus_queue.cc:237] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.027915 18427 raft_consensus.cc:493] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.028403 18427 raft_consensus.cc:515] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } }
I20260709 13:18:24.032785 18427 leader_election.cc:290] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.035032 11514 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:24.038349 11293 catalog_manager.cc:5697] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b reported cstate change: term changed from 18 to 20, leader changed from b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4) to 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3). New cstate: current_term: 20 leader_uuid: "9e9c2603c0bd4d25b713649b3890c46b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:24.041145 11516 leader_election.cc:336] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:24.041558 11516 leader_election.cc:304] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:24.042155 18427 raft_consensus.cc:2749] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:24.053287 12085 consensus_peers.cc:597] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.058599 11545 raft_consensus.cc:1275] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:24.060117 18405 consensus_queue.cc:1048] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:24.077483 10252 tablet_server.cc:196] TabletServer@127.10.3.2:0 shutdown complete.
W20260709 13:18:24.077425 12085 consensus_peers.cc:597] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.084214 12083 consensus_peers.cc:597] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.086410 18407 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.086956 18407 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.088598 18407 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:24.090502 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b" is_pre_election: true
I20260709 13:18:24.091189 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 20.
W20260709 13:18:24.093178 12085 consensus_peers.cc:597] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.094281 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 
I20260709 13:18:24.095763 18407 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:18:24.096141 18407 raft_consensus.cc:493] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.096482 18407 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:24.097517 12083 consensus_peers.cc:597] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.103900 18407 raft_consensus.cc:515] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.105813 11545 raft_consensus.cc:1275] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 13:18:24.106658 18407 leader_election.cc:290] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 election: Requested vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:24.108613 11545 raft_consensus.cc:1275] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Refusing update from remote peer b3abf4b5ee724203ad1226f5c46bfe46: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:24.110630 18407 consensus_queue.cc:1048] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:18:24.111020 18405 consensus_queue.cc:1048] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:18:24.113910 11545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e90063696c0437bb8ad2d545f45a9ce" candidate_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9e9c2603c0bd4d25b713649b3890c46b"
I20260709 13:18:24.114516 11545 raft_consensus.cc:3060] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:18:24.115182 12085 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:24.122169 11545 raft_consensus.cc:2468] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3abf4b5ee724203ad1226f5c46bfe46 in term 21.
W20260709 13:18:24.125771 12085 leader_election.cc:336] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:24.127115 12085 leader_election.cc:304] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e9c2603c0bd4d25b713649b3890c46b, b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892
I20260709 13:18:24.128187 18405 raft_consensus.cc:2804] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:18:24.129524 18405 raft_consensus.cc:697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 LEADER]: Becoming Leader. State: Replica: b3abf4b5ee724203ad1226f5c46bfe46, State: Running, Role: LEADER
I20260709 13:18:24.130463 18405 consensus_queue.cc:237] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.139534 12119 raft_consensus.cc:1275] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:18:24.141155 18427 consensus_queue.cc:1048] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:18:24.186335 11514 consensus_peers.cc:597] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Couldn't send request to peer 34fe1737ebb840fb9746980f649cc0a6. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.186100 11293 catalog_manager.cc:5697] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 reported cstate change: term changed from 19 to 21, leader changed from 147201d22a694ca59f6e014f083a0892 (127.10.3.1) to b3abf4b5ee724203ad1226f5c46bfe46 (127.10.3.4). New cstate: current_term: 21 leader_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:24.190820 11516 consensus_peers.cc:597] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.220726 10252 tablet_server.cc:179] TabletServer@127.10.3.3:0 shutting down...
W20260709 13:18:24.228303 12085 consensus_peers.cc:597] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.258455 12119 raft_consensus.cc:1275] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Refusing update from remote peer 9e9c2603c0bd4d25b713649b3890c46b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:18:24.260912 12085 consensus_peers.cc:597] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.261373 18427 consensus_queue.cc:1048] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:18:24.262668 11516 consensus_peers.cc:597] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.269253 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:24.270377 10252 tablet_replica.cc:333] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.271211 10252 raft_consensus.cc:2243] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.271891 10252 raft_consensus.cc:2272] T 058fa5ec3edb42c796103c7361c1a8e1 P 9e9c2603c0bd4d25b713649b3890c46b [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.275327 10252 tablet_replica.cc:333] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.276117 10252 raft_consensus.cc:2243] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.276772 10252 raft_consensus.cc:2272] T 8e90063696c0437bb8ad2d545f45a9ce P 9e9c2603c0bd4d25b713649b3890c46b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.279043 10252 tablet_replica.cc:333] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.279704 10252 raft_consensus.cc:2243] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.280488 10252 raft_consensus.cc:2272] T 13474bcfb1f1443f82e90db3445bc880 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.282722 10252 tablet_replica.cc:333] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.283308 10252 raft_consensus.cc:2243] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.283839 10252 raft_consensus.cc:2272] T 66cb4598d33043fc8eb235647a242814 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.286495 10252 tablet_replica.cc:333] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.287297 10252 raft_consensus.cc:2243] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.287943 10252 raft_consensus.cc:2272] T 95e3d59427ee46f989532eb3cdc311cc P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:24.289824 12085 consensus_peers.cc:597] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.290531 10252 tablet_replica.cc:333] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.291270 10252 raft_consensus.cc:2243] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.291759 10252 raft_consensus.cc:2272] T 778170f25cb24e08b57cbbc852e8a007 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.294852 10252 tablet_replica.cc:333] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.295511 10252 raft_consensus.cc:2243] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.296326 10252 raft_consensus.cc:2272] T ff56d6db2ff44eb19771b9166c042077 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.298199 10252 tablet_replica.cc:333] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.298750 10252 raft_consensus.cc:2243] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.299201 10252 raft_consensus.cc:2272] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.301046 10252 tablet_replica.cc:333] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.301570 10252 raft_consensus.cc:2243] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.302006 10252 raft_consensus.cc:2272] T c3fd80658464474dbe463b1943becc75 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.303925 10252 tablet_replica.cc:333] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.304433 10252 raft_consensus.cc:2243] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.304939 10252 raft_consensus.cc:2272] T a6454d815ae44cdda2fe8d013fca9b4b P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.307520 10252 tablet_replica.cc:333] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.308203 10252 raft_consensus.cc:2243] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.308694 10252 raft_consensus.cc:2272] T 71ff2b5fd08d4882a5355cf7758991cf P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:24.310930 12085 consensus_peers.cc:597] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.311111 10252 tablet_replica.cc:333] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.311863 10252 raft_consensus.cc:2243] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.312253 10252 raft_consensus.cc:2272] T 56b606c6942f42cf914bcabc7e2bc502 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.313997 10252 tablet_replica.cc:333] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.314472 10252 raft_consensus.cc:2243] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.315280 10252 raft_consensus.cc:2272] T 59af340b26264692af5490466dd69b78 P 9e9c2603c0bd4d25b713649b3890c46b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.317026 10252 tablet_replica.cc:333] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.317507 10252 raft_consensus.cc:2243] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.317916 10252 raft_consensus.cc:2272] T 902c85625c4647d481fa8ef07be336f4 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.320143 10252 tablet_replica.cc:333] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.320684 10252 raft_consensus.cc:2243] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.321347 10252 raft_consensus.cc:2272] T 79d2059876e34ff89bf967e96ab7bf09 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.323462 10252 tablet_replica.cc:333] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.324007 10252 raft_consensus.cc:2243] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.324477 10252 raft_consensus.cc:2272] T a3205d9e6dce4aad891fa90943f21879 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.326319 10252 tablet_replica.cc:333] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.326848 10252 raft_consensus.cc:2243] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.327389 10252 raft_consensus.cc:2272] T 1f946b7973fe482681135635493ebd48 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.329118 10252 tablet_replica.cc:333] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.329638 10252 raft_consensus.cc:2243] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.330034 10252 raft_consensus.cc:2272] T a11e4197bc8843c69871c3f11dc98474 P 9e9c2603c0bd4d25b713649b3890c46b [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.332257 10252 tablet_replica.cc:333] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.332893 10252 raft_consensus.cc:2243] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.333654 10252 raft_consensus.cc:2272] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.335556 10252 tablet_replica.cc:333] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.336089 10252 raft_consensus.cc:2243] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.336547 10252 raft_consensus.cc:2272] T 255638a15529487e8ade55aaaa8455d5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.338728 10252 tablet_replica.cc:333] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.339316 10252 raft_consensus.cc:2243] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.340095 10252 raft_consensus.cc:2272] T ffe5740695024e83afc5069cfb41118c P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.341967 10252 tablet_replica.cc:333] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.342491 10252 raft_consensus.cc:2243] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.342908 10252 raft_consensus.cc:2272] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.344992 10252 tablet_replica.cc:333] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.345538 10252 raft_consensus.cc:2243] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.346302 10252 raft_consensus.cc:2272] T 4e67029edeb04a569eb2c35676ef78da P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.348253 10252 tablet_replica.cc:333] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.348789 10252 raft_consensus.cc:2243] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.349303 10252 raft_consensus.cc:2272] T b5de422ef4884189bea49cec7b7c2ec1 P 9e9c2603c0bd4d25b713649b3890c46b [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.351526 10252 tablet_replica.cc:333] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.352191 10252 raft_consensus.cc:2243] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.353140 10252 raft_consensus.cc:2272] T b66bfe782a6d489fa35dc079b8c4d0a5 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.355123 10252 tablet_replica.cc:333] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.355672 10252 raft_consensus.cc:2243] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.356118 10252 raft_consensus.cc:2272] T aa999fcc28454bc88f93770d07ff72e2 P 9e9c2603c0bd4d25b713649b3890c46b [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.357897 10252 tablet_replica.cc:333] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.358428 10252 raft_consensus.cc:2243] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.358873 10252 raft_consensus.cc:2272] T 48129c3807a242fca6439f00aefd9b22 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.360618 10252 tablet_replica.cc:333] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.361128 10252 raft_consensus.cc:2243] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.361922 10252 raft_consensus.cc:2272] T b2eeda82ae3942d6bf0db873fa350bdc P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.364002 10252 tablet_replica.cc:333] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.364521 10252 raft_consensus.cc:2243] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.364955 10252 raft_consensus.cc:2272] T 5e9e663b46c5417b9a0e42fdc7b70969 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.367084 10252 tablet_replica.cc:333] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.367678 10252 raft_consensus.cc:2243] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.368152 10252 raft_consensus.cc:2272] T 840a1b524ac54acdb12289fa1aafbd17 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:24.368927 12085 consensus_peers.cc:597] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.370501 10252 tablet_replica.cc:333] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.371178 10252 raft_consensus.cc:2243] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.371702 10252 raft_consensus.cc:2272] T 930d965be7eb49448fa23456beb9925a P 9e9c2603c0bd4d25b713649b3890c46b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.373505 10252 tablet_replica.cc:333] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.374024 10252 raft_consensus.cc:2243] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.374446 10252 raft_consensus.cc:2272] T 6bea8de48c2942d99a1477b5b427be03 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.376237 10252 tablet_replica.cc:333] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.376861 10252 raft_consensus.cc:2243] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.377532 10252 pending_rounds.cc:70] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Trying to abort 1 pending ops.
I20260709 13:18:24.377734 10252 pending_rounds.cc:77] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305638314100883456 op_type: NO_OP noop_request { }
I20260709 13:18:24.378022 10252 raft_consensus.cc:2889] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:24.378304 10252 raft_consensus.cc:2272] T fe707c46816d4d84bf4ead0d04de2b26 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.380436 10252 tablet_replica.cc:333] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.380959 10252 raft_consensus.cc:2243] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.381422 10252 raft_consensus.cc:2272] T 9b37b6683af04823b0814ba56c4ea803 P 9e9c2603c0bd4d25b713649b3890c46b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.383391 10252 tablet_replica.cc:333] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.383876 10252 raft_consensus.cc:2243] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:24.383922 12085 consensus_peers.cc:597] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.384482 10252 raft_consensus.cc:2272] T f52a477150c345c8ba80211d1a49b41c P 9e9c2603c0bd4d25b713649b3890c46b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.386263 10252 tablet_replica.cc:333] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.386792 10252 raft_consensus.cc:2243] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.387233 10252 raft_consensus.cc:2272] T 43c1c5858a0f4571a21e2d097910aa44 P 9e9c2603c0bd4d25b713649b3890c46b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.389128 10252 tablet_replica.cc:333] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.389647 10252 raft_consensus.cc:2243] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.390061 10252 raft_consensus.cc:2272] T 25be57d1c24d4a43859ce0b5848adc09 P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.392441 10252 tablet_replica.cc:333] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.392961 10252 raft_consensus.cc:2243] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:24.393055 12085 consensus_peers.cc:597] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:24.393462 10252 raft_consensus.cc:2272] T 3c4de37bdc334587a90c03b40bfd6c5f P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.395370 10252 tablet_replica.cc:333] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.395896 10252 raft_consensus.cc:2243] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.396322 10252 raft_consensus.cc:2272] T e8e0fc54e8b645deb82c6013c5d5c7ea P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.398057 10252 tablet_replica.cc:333] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.398564 10252 raft_consensus.cc:2243] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.399047 10252 raft_consensus.cc:2272] T 6d02abb4f52449679a2415346f354340 P 9e9c2603c0bd4d25b713649b3890c46b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.400878 10252 tablet_replica.cc:333] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.401360 10252 raft_consensus.cc:2243] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.402122 10252 raft_consensus.cc:2272] T eeb668f66ce344e9acdf8834c2b1d3a3 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.402797 18407 raft_consensus.cc:493] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.403456 18407 raft_consensus.cc:515] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.404537 10252 tablet_replica.cc:333] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.405200 10252 raft_consensus.cc:2243] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.405316 18407 leader_election.cc:290] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:24.405857 10252 raft_consensus.cc:2272] T 5b22c9e294a846159d4831b92f692943 P 9e9c2603c0bd4d25b713649b3890c46b [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.408186 10252 tablet_replica.cc:333] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
W20260709 13:18:24.408492 12085 leader_election.cc:336] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:24.408874 10252 raft_consensus.cc:2243] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.409415 10252 raft_consensus.cc:2272] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P 9e9c2603c0bd4d25b713649b3890c46b [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:24.409548 12083 leader_election.cc:336] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:24.410593 12083 leader_election.cc:304] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
I20260709 13:18:24.411687 10252 tablet_replica.cc:333] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b: stopping tablet replica
I20260709 13:18:24.412204 10252 raft_consensus.cc:2243] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.412796 18407 raft_consensus.cc:2749] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.412967 10252 raft_consensus.cc:2272] T b150719d534f439aa33eb70af414a5d0 P 9e9c2603c0bd4d25b713649b3890c46b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.419598 18407 raft_consensus.cc:493] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.420048 18407 raft_consensus.cc:515] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.421777 18407 leader_election.cc:290] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.426146 12085 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:24.426667 12083 leader_election.cc:336] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Network error: Client connection negotiation failed: client connection to 127.10.3.2:36161: connect: Connection refused (error 111)
I20260709 13:18:24.427085 12083 leader_election.cc:304] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
I20260709 13:18:24.427688 18407 raft_consensus.cc:2749] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.439630 18407 raft_consensus.cc:493] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.440172 18407 raft_consensus.cc:515] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.442523 18407 leader_election.cc:290] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:24.445385 12085 leader_election.cc:336] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:24.456254 18407 raft_consensus.cc:493] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.456885 18407 raft_consensus.cc:515] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.459263 18407 leader_election.cc:290] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.464234 12085 leader_election.cc:336] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:24.465776 12085 consensus_peers.cc:597] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.468567 12085 consensus_peers.cc:597] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.484078 18407 raft_consensus.cc:493] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.485343 18407 raft_consensus.cc:515] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.487990 18407 leader_election.cc:290] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.493131 12085 leader_election.cc:336] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:24.515233 18407 raft_consensus.cc:493] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.515892 18407 raft_consensus.cc:515] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.518100 18407 leader_election.cc:290] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.521472 12085 leader_election.cc:336] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:24.550293 12085 consensus_peers.cc:597] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.569288 18407 raft_consensus.cc:493] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.569875 18407 raft_consensus.cc:515] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.572046 18407 leader_election.cc:290] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
I20260709 13:18:24.572467 18405 raft_consensus.cc:493] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.573064 18405 raft_consensus.cc:515] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.575501 18405 leader_election.cc:290] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:24.577214 12085 leader_election.cc:336] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111)
I20260709 13:18:24.579430 18405 raft_consensus.cc:493] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.579926 18405 raft_consensus.cc:515] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
W20260709 13:18:24.580855 12085 leader_election.cc:336] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111)
I20260709 13:18:24.582320 18405 leader_election.cc:290] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.586510 12085 leader_election.cc:336] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
W20260709 13:18:24.593532 12085 consensus_peers.cc:597] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.614418 18405 raft_consensus.cc:493] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.615078 18405 raft_consensus.cc:515] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.617038 18405 leader_election.cc:290] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:24.621006 12085 leader_election.cc:336] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111)
W20260709 13:18:24.679378 12085 consensus_peers.cc:597] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.693570 12085 consensus_peers.cc:597] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.727109 12085 consensus_peers.cc:597] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.729656 12085 consensus_peers.cc:597] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Couldn't send request to peer 147201d22a694ca59f6e014f083a0892. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.750979 12085 consensus_peers.cc:597] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 -> Peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Couldn't send request to peer 9e9c2603c0bd4d25b713649b3890c46b. Status: Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.791055 18405 raft_consensus.cc:493] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.791666 18405 raft_consensus.cc:515] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.793439 18405 leader_election.cc:290] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:24.797240 12085 leader_election.cc:336] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:24.813452 18405 raft_consensus.cc:493] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34fe1737ebb840fb9746980f649cc0a6)
I20260709 13:18:24.814042 18405 raft_consensus.cc:515] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "9e9c2603c0bd4d25b713649b3890c46b" member_type: VOTER last_known_addr { host: "127.10.3.3" port: 46781 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:24.816009 18405 leader_election.cc:290] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161), 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781)
W20260709 13:18:24.819725 12085 leader_election.cc:336] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9e9c2603c0bd4d25b713649b3890c46b (127.10.3.3:46781): Network error: Client connection negotiation failed: client connection to 127.10.3.3:46781: connect: Connection refused (error 111)
I20260709 13:18:25.005748 18405 raft_consensus.cc:493] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 147201d22a694ca59f6e014f083a0892)
I20260709 13:18:25.006336 18405 raft_consensus.cc:515] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:25.008507 18405 leader_election.cc:290] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:25.011888 12085 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:25.118496 10252 tablet_server.cc:196] TabletServer@127.10.3.3:0 shutdown complete.
I20260709 13:18:25.213578 10252 tablet_server.cc:179] TabletServer@127.10.3.4:0 shutting down...
I20260709 13:18:25.222330 18405 raft_consensus.cc:493] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.222889 18405 raft_consensus.cc:515] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:25.226740 18405 leader_election.cc:290] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:25.231688 12085 leader_election.cc:336] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:25.239519 18405 raft_consensus.cc:493] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.240247 18405 raft_consensus.cc:515] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:25.242306 18405 leader_election.cc:290] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
W20260709 13:18:25.248188 12085 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:25.258818 10252 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:25.259744 10252 tablet_replica.cc:333] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.260396 10252 raft_consensus.cc:2243] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.260860 10252 raft_consensus.cc:2272] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.262820 10252 tablet_replica.cc:333] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.263454 10252 raft_consensus.cc:2243] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.264014 10252 raft_consensus.cc:2272] T 4e67029edeb04a569eb2c35676ef78da P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.265843 10252 tablet_replica.cc:333] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.266433 10252 raft_consensus.cc:2243] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.267274 10252 raft_consensus.cc:2272] T 25be57d1c24d4a43859ce0b5848adc09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.269202 10252 tablet_replica.cc:333] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.269972 10252 raft_consensus.cc:2243] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.270538 10252 raft_consensus.cc:2272] T eeb668f66ce344e9acdf8834c2b1d3a3 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.273675 10252 tablet_replica.cc:333] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.274399 10252 raft_consensus.cc:2243] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.274860 18406 raft_consensus.cc:493] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.275076 10252 raft_consensus.cc:2272] T 38b39fd56a8d45f7b56a9ac114b66d2b P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.275338 18406 raft_consensus.cc:515] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "147201d22a694ca59f6e014f083a0892" member_type: VOTER last_known_addr { host: "127.10.3.1" port: 37459 } } peers { permanent_uuid: "34fe1737ebb840fb9746980f649cc0a6" member_type: VOTER last_known_addr { host: "127.10.3.2" port: 36161 } } peers { permanent_uuid: "b3abf4b5ee724203ad1226f5c46bfe46" member_type: VOTER last_known_addr { host: "127.10.3.4" port: 45993 } attrs { promote: false } }
I20260709 13:18:25.277580 18406 leader_election.cc:290] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459), 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161)
I20260709 13:18:25.278029 10252 tablet_replica.cc:333] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.279050 10252 raft_consensus.cc:2243] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.279589 10252 raft_consensus.cc:2272] T b2dfc6e65bda4ed2a39f4bce45ec85e6 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:25.281579 12085 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 147201d22a694ca59f6e014f083a0892 (127.10.3.1:37459): Network error: Client connection negotiation failed: client connection to 127.10.3.1:37459: connect: Connection refused (error 111)
I20260709 13:18:25.282140 10252 tablet_replica.cc:333] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.282761 10252 raft_consensus.cc:2243] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.283650 10252 raft_consensus.cc:2272] T 930d965be7eb49448fa23456beb9925a P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.285930 10252 tablet_replica.cc:333] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.286664 10252 raft_consensus.cc:2243] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.287248 10252 raft_consensus.cc:2272] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.289183 10252 tablet_replica.cc:333] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.289686 10252 raft_consensus.cc:2243] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.290337 10252 raft_consensus.cc:2272] T 12c14d0fd7ef4fbf899a4b3eebe33e5b P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.292127 10252 tablet_replica.cc:333] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.292634 10252 raft_consensus.cc:2243] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.293262 10252 raft_consensus.cc:2272] T b61ac66d97f84c1186c6e2b1d79477b2 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.295083 10252 tablet_replica.cc:333] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.295647 10252 raft_consensus.cc:2243] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.296244 10252 raft_consensus.cc:2272] T 74bea11400ce497283dceb7d21b6e728 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.299026 10252 tablet_replica.cc:333] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.299973 10252 raft_consensus.cc:2243] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.300458 10252 raft_consensus.cc:2272] T 13474bcfb1f1443f82e90db3445bc880 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.302498 10252 tablet_replica.cc:333] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.303112 10252 raft_consensus.cc:2243] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.303572 10252 raft_consensus.cc:2272] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.305306 10252 tablet_replica.cc:333] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.305816 10252 raft_consensus.cc:2243] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.306537 10252 pending_rounds.cc:70] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Trying to abort 1 pending ops.
I20260709 13:18:25.306732 10252 pending_rounds.cc:77] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46: Aborting op as it isn't in flight: id { term: 21 index: 16 } timestamp: 7305638314554597376 op_type: NO_OP noop_request { }
I20260709 13:18:25.307040 10252 raft_consensus.cc:2889] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:25.307314 10252 raft_consensus.cc:2272] T 8e90063696c0437bb8ad2d545f45a9ce P b3abf4b5ee724203ad1226f5c46bfe46 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.308998 10252 tablet_replica.cc:333] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.309517 10252 raft_consensus.cc:2243] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.309902 10252 raft_consensus.cc:2272] T b2eeda82ae3942d6bf0db873fa350bdc P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.311662 10252 tablet_replica.cc:333] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.312177 10252 raft_consensus.cc:2243] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.312604 10252 raft_consensus.cc:2272] T 5e9e663b46c5417b9a0e42fdc7b70969 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.314275 10252 tablet_replica.cc:333] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.314760 10252 raft_consensus.cc:2243] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.315243 10252 raft_consensus.cc:2272] T b66bfe782a6d489fa35dc079b8c4d0a5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.316937 10252 tablet_replica.cc:333] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.317530 10252 raft_consensus.cc:2243] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.318239 10252 raft_consensus.cc:2272] T 6eba703fc47843af924d4e45f12db16b P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.320058 10252 tablet_replica.cc:333] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.320576 10252 raft_consensus.cc:2243] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.320998 10252 raft_consensus.cc:2272] T 56b606c6942f42cf914bcabc7e2bc502 P b3abf4b5ee724203ad1226f5c46bfe46 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.322603 10252 tablet_replica.cc:333] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.323171 10252 raft_consensus.cc:2243] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.323599 10252 raft_consensus.cc:2272] T 058fa5ec3edb42c796103c7361c1a8e1 P b3abf4b5ee724203ad1226f5c46bfe46 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.325352 10252 tablet_replica.cc:333] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.325912 10252 raft_consensus.cc:2243] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.326431 10252 raft_consensus.cc:2272] T 59af340b26264692af5490466dd69b78 P b3abf4b5ee724203ad1226f5c46bfe46 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.328217 10252 tablet_replica.cc:333] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.328725 10252 raft_consensus.cc:2243] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.329430 10252 raft_consensus.cc:2272] T 902c85625c4647d481fa8ef07be336f4 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.331355 10252 tablet_replica.cc:333] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.331904 10252 raft_consensus.cc:2243] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.332440 10252 raft_consensus.cc:2272] T b8eacd98a9dc4961a9fac3d2ae9dbd02 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.334283 10252 tablet_replica.cc:333] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.334818 10252 raft_consensus.cc:2243] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.335609 10252 raft_consensus.cc:2272] T 3c4de37bdc334587a90c03b40bfd6c5f P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.337468 10252 tablet_replica.cc:333] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.337999 10252 raft_consensus.cc:2243] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.338752 10252 raft_consensus.cc:2272] T 6d02abb4f52449679a2415346f354340 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.340576 10252 tablet_replica.cc:333] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.341097 10252 raft_consensus.cc:2243] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.341753 10252 raft_consensus.cc:2272] T a6454d815ae44cdda2fe8d013fca9b4b P b3abf4b5ee724203ad1226f5c46bfe46 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.343539 10252 tablet_replica.cc:333] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.344069 10252 raft_consensus.cc:2243] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.344457 10252 pending_rounds.cc:70] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Trying to abort 1 pending ops.
I20260709 13:18:25.344646 10252 pending_rounds.cc:77] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: Aborting op as it isn't in flight: id { term: 23 index: 17 } timestamp: 7305638286840029184 op_type: NO_OP noop_request { }
I20260709 13:18:25.344910 10252 raft_consensus.cc:2889] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:25.345175 10252 raft_consensus.cc:2272] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.346935 10252 tablet_replica.cc:333] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.347534 10252 raft_consensus.cc:2243] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.348476 10252 raft_consensus.cc:2272] T 840a1b524ac54acdb12289fa1aafbd17 P b3abf4b5ee724203ad1226f5c46bfe46 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.351058 10252 tablet_replica.cc:333] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.351718 10252 raft_consensus.cc:2243] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.352386 10252 raft_consensus.cc:2272] T e106bf87136140b4bebad713d83d02fe P b3abf4b5ee724203ad1226f5c46bfe46 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.354673 10252 tablet_replica.cc:333] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.355341 10252 raft_consensus.cc:2243] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.355788 10252 raft_consensus.cc:2272] T 7abd40007d144575bd9e785e609a62df P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.357506 10252 tablet_replica.cc:333] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.358008 10252 raft_consensus.cc:2243] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.358819 10252 raft_consensus.cc:2272] T 6a6ebf03c40d4f1fa404e0fe9f8f9fc5 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.360811 10252 tablet_replica.cc:333] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.361346 10252 raft_consensus.cc:2243] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.361775 10252 raft_consensus.cc:2272] T 79d2059876e34ff89bf967e96ab7bf09 P b3abf4b5ee724203ad1226f5c46bfe46 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.363622 10252 tablet_replica.cc:333] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.364192 10252 raft_consensus.cc:2243] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.364598 10252 raft_consensus.cc:2272] T b150719d534f439aa33eb70af414a5d0 P b3abf4b5ee724203ad1226f5c46bfe46 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.366315 10252 tablet_replica.cc:333] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.366824 10252 raft_consensus.cc:2243] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.367302 10252 raft_consensus.cc:2272] T 9b37b6683af04823b0814ba56c4ea803 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.369110 10252 tablet_replica.cc:333] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.369601 10252 raft_consensus.cc:2243] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.370260 10252 raft_consensus.cc:2272] T cfd1f78aa1d843c28021b808cf5fd960 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.372056 10252 tablet_replica.cc:333] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.372565 10252 raft_consensus.cc:2243] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.372972 10252 raft_consensus.cc:2272] T 4f615bf163b9483d8209eb6c8fedca15 P b3abf4b5ee724203ad1226f5c46bfe46 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.374828 10252 tablet_replica.cc:333] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.375365 10252 raft_consensus.cc:2243] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.375809 10252 raft_consensus.cc:2272] T dba44ee610b54495a5f1cfea0a876524 P b3abf4b5ee724203ad1226f5c46bfe46 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.377564 10252 tablet_replica.cc:333] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.378077 10252 raft_consensus.cc:2243] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.378468 10252 raft_consensus.cc:2272] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.380373 10252 tablet_replica.cc:333] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.380878 10252 raft_consensus.cc:2243] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.381379 10252 raft_consensus.cc:2272] T 1c4eb2d16aa247998d0927eb34dedfa8 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:25.381945 12083 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.383528 10252 tablet_replica.cc:333] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.384043 10252 raft_consensus.cc:2243] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.384646 10252 raft_consensus.cc:2272] T a11e4197bc8843c69871c3f11dc98474 P b3abf4b5ee724203ad1226f5c46bfe46 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.386277 10252 tablet_replica.cc:333] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.386700 10252 raft_consensus.cc:2243] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.387125 10252 raft_consensus.cc:2272] T ff56d6db2ff44eb19771b9166c042077 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.388865 10252 tablet_replica.cc:333] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.389300 10252 raft_consensus.cc:2243] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.389653 10252 raft_consensus.cc:2272] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.391260 10252 tablet_replica.cc:333] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.391746 10252 raft_consensus.cc:2243] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.392141 10252 raft_consensus.cc:2272] T fe707c46816d4d84bf4ead0d04de2b26 P b3abf4b5ee724203ad1226f5c46bfe46 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.393887 10252 tablet_replica.cc:333] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46: stopping tablet replica
I20260709 13:18:25.394271 10252 raft_consensus.cc:2243] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.394836 10252 raft_consensus.cc:2272] T 778170f25cb24e08b57cbbc852e8a007 P b3abf4b5ee724203ad1226f5c46bfe46 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:25.472719 18365 leader_election.cc:336] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.473177 18365 leader_election.cc:304] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:25.473779 18365 raft_consensus.cc:2686] T 08ebf4312af74188830a07342d0dc119 P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:18:25.475638 18365 leader_election.cc:336] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.476001 18365 leader_election.cc:304] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:25.476745 18365 raft_consensus.cc:2686] T 44539667cbc04657b801ff659dee5bde P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:18:25.480569 18365 leader_election.cc:336] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.480947 18365 leader_election.cc:304] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:25.481434 18365 raft_consensus.cc:2686] T 5e8ed42aaa39460bbea1f8deeccd0adc P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: The pool has been shut down.
W20260709 13:18:25.482888 18365 leader_election.cc:336] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.483276 18365 leader_election.cc:304] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:25.483716 18365 raft_consensus.cc:2686] T c5a7fd6f89ac4da3ab7769d1379f5bb8 P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: The pool has been shut down.
W20260709 13:18:25.485492 18365 leader_election.cc:336] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.485817 18365 leader_election.cc:304] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 34fe1737ebb840fb9746980f649cc0a6, 9e9c2603c0bd4d25b713649b3890c46b
W20260709 13:18:25.486254 18365 raft_consensus.cc:2686] T 43c1c5858a0f4571a21e2d097910aa44 P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: The pool has been shut down.
W20260709 13:18:25.487749 18365 leader_election.cc:336] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 34fe1737ebb840fb9746980f649cc0a6 (127.10.3.2:36161): Service unavailable: reactor is shutting down (error 108)
I20260709 13:18:25.488091 18365 leader_election.cc:304] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3abf4b5ee724203ad1226f5c46bfe46; no voters: 147201d22a694ca59f6e014f083a0892, 34fe1737ebb840fb9746980f649cc0a6
W20260709 13:18:25.488531 18365 raft_consensus.cc:2686] T b10b16798ae24794b186d5daa625c6d1 P b3abf4b5ee724203ad1226f5c46bfe46: unable to run election callback: Service unavailable: The pool has been shut down.
F20260709 13:18:25.490239 10252 leader_election.cc:206] Check failed: has_responded_ 
*** Check failure stack trace: ***
*** Aborted at 1783603105 (unix time) try "date -d @1783603105" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e80000280c) received by PID 10252 (TID 0x7f8da245f840) from PID 10252; stack trace: ***
    @           0x2d8530 __tsan::CallUserSignalHandler() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
    @           0x2daa13 rtl_sigaction() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
    @     0x7f8da9d21980 (unknown) at ??:0
    @     0x7f8da311fe87 gsignal at ??:0
    @     0x7f8da31217f1 abort at ??:0
    @           0x2de526 abort at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
    @     0x7f8da5ef1d84 google::LogMessage::SendToLog() at ??:0
    @     0x7f8da5ef2930 google::LogMessage::Flush() at ??:0
    @     0x7f8da5ef7a6b google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f8da98c5c9c kudu::consensus::LeaderElection::~LeaderElection() at ??:0
    @     0x7f8da98d34df kudu::RefCountedThreadSafe<>::DeleteInternal() at ??:0
    @     0x7f8da98d34aa kudu::DefaultRefCountedThreadSafeTraits<>::Destruct() at ??:0
    @     0x7f8da98d3466 kudu::RefCountedThreadSafe<>::Release() at ??:0
    @     0x7f8da98caf37 scoped_refptr<>::~scoped_refptr() at ??:0
    @     0x7f8da98c72d3 kudu::consensus::LeaderElection::Run()::$_0::~Run() at ??:0
    @     0x7f8da98c8eba std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f8da98c97da std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f8da98c8aab std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x37d8bb std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x3671da std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
    @     0x7f8da7be923e kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest()::$_0::~RefreshDnsAndEnqueueRequest() at ??:0
    @     0x7f8da7bea5fa std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f8da7beaf2a std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f8da7bea1cb std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @     0x7f8daf72f64b std::__1::__function::__value_func<>::~__value_func() at ??:0
    @     0x7f8daf7006ea std::__1::function<>::~function() at ??:0
    @     0x7f8da67a7f6b kudu::DnsResolver::RefreshAddressesAsync()::$_1::~RefreshAddressesAsync() at ??:0
    @     0x7f8da67aa16a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f8da67aaada std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f8da67a9d0b std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x37d8bb std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x3671da std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
